nV News Forums

 
 

nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   General Linux (http://www.nvnews.net/vbulletin/forumdisplay.php?f=27)
-   -   CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67) (http://www.nvnews.net/vbulletin/showthread.php?t=134655)

paix 06-18-09 10:52 AM

CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Hi all.

I have a strange problems with the network card.
One time I have got a kernel panic.
You could see screenshot from IPKVM here: http://paix.org.ua/tmp/panik_260509.jpg

Sometimes under high network load my server become unavailable, but trough IPKVM server works fine, and after
Code:

service network restart
netcard start work again.

Very often under high network load too my server become pings with a very long time.
Code:

64 bytes from xxx.xxx: icmp_seq=71 ttl=56 time=2218 ms
64 bytes from xxx.xxx: icmp_seq=72 ttl=56 time=2208 ms
64 bytes from xxx.xxx: icmp_seq=74 ttl=56 time=1047 ms

Server connected by 100mbit link, and I ping it from neighbour machine.
And after
Code:

service network restart
server come in to normal behavior.

I am running openvz kernel, which is based on the current RHEL5 kernel.

Code:

#uname -a
Linux domain 2.6.18-128.1.1.el5.028stab062.3 #1 SMP Sun May 10 18:54:51 MSD 2009 x86_64 x86_64 x86_64 GNU/Linux

Code:

Base Board Information
        Manufacturer: ASUSTeK Computer INC.
        Product Name: M2N-VM DVI

# from dmesg:
Code:

00:0a.0 Ethernet controller: nVidia Corporation MCP67 Ethernet (rev a2)
forcedeth.c: Reverse Engineered nForce ethernet driver. Version 0.60.
forcedeth: using HIGHDMA
eth0: forcedeth.c: subsystem: 01043:82b3 bound to 0000:00:0a.0

Code:

# lspci
00:00.0 RAM memory: nVidia Corporation MCP67 Memory Controller (rev a2)
00:01.0 ISA bridge: nVidia Corporation MCP67 ISA Bridge (rev a2)
00:01.1 SMBus: nVidia Corporation MCP67 SMBus (rev a2)
00:02.0 USB Controller: nVidia Corporation MCP67 OHCI USB 1.1 Controller (rev a2)
00:02.1 USB Controller: nVidia Corporation MCP67 EHCI USB 2.0 Controller (rev a2)
00:04.0 USB Controller: nVidia Corporation MCP67 OHCI USB 1.1 Controller (rev a2)
00:04.1 USB Controller: nVidia Corporation MCP67 EHCI USB 2.0 Controller (rev a2)
00:06.0 IDE interface: nVidia Corporation MCP67 IDE Controller (rev a1)
00:07.0 Audio device: nVidia Corporation MCP67 High Definition Audio (rev a1)
00:08.0 PCI bridge: nVidia Corporation MCP67 PCI Bridge (rev a2)
00:09.0 IDE interface: nVidia Corporation MCP67 AHCI Controller (rev a2)
00:0a.0 Ethernet controller: nVidia Corporation MCP67 Ethernet (rev a2)
00:0b.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:0c.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:0d.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:0e.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:0f.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:10.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:11.0 PCI bridge: nVidia Corporation MCP67 PCI Express Bridge (rev a2)
00:12.0 VGA compatible controller: nVidia Corporation GeForce 7050 PV / nForce 630a (rev a2)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control

Any advice will be greatly appreciated!
Thanks!

paix 06-18-09 10:55 AM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Sorry, I'm running CentOS 5.3, not 5.2

whig 06-18-09 05:59 PM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
You can restart the network, can you run "top" to see if any processes are using 99% cpu/ram/swap?

paix 06-19-09 03:13 AM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Quote:

Originally Posted by whig (Post 2032191)
You can restart the network, can you run "top" to see if any processes are using 99% cpu/ram/swap?

There are no cpu\ram\disk intensive process when netcard become unavailable (and during time when netcard not responding).
Also there are no any messages about this in log/messages or dmesg.

paix 06-19-09 08:09 AM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Recently I have got too panics, when I tested the network by iperf.

kernel booted with
Code:

irqpoll nousb noapic
http://paix.org.ua/tmp/panic_190609.jpg

kernel booted with
Code:

nousb noapic
and

Code:

alias eth0 forcedeth
options forcedeth optimization_mode=1

http://paix.org.ua/tmp/panic2_190609.jpg

Also there is one interesting oops in log/messages:
Code:

kernel: skb_over_panic: text:ffffffff881bf46f len:15398 put:15398 head:ffff8100a25c5800 data:ffff8100a25c5810 tail:ffff8100a25c9436 end:ffff8100a25c5e80 dev:eth0
kernel: ----------- [cut here ] --------- [please bite here ] ---------
kernel: Kernel BUG at net/core/skbuff.c:96

Also I have a kernel booted with crashkernel=128M@16M option, and I have kdump running, but unfortunately there are no any saved core. I couldn't reboot server via ipkvm too, so I've requested a support to hardware reboot the server.

dmesg:
http://paix.org.ua/tmp/dmesg_190609.txt

whig 06-20-09 06:18 PM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Post the results of this command in the problem condition
Code:

top -b -n 1

paix 06-22-09 06:39 AM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Quote:

Originally Posted by whig (Post 2033329)
Post the results of this command in the problem condition
Code:

top -b -n 1

the NIC completely freeze a server. I even can't reboot the server through IPKVM :(

I've stressed the NIC today by iperf package (from epel. Description: Iperf is a tool to measure maximum TCP bandwidth) and got the panic.

The screenshot from IPKVM here: http://paix.org.ua/tmp/panic_220609.jpg
Unfortunately there isn't any interesting info to identify the problem.

The kernel was loaded with

Code:

kernel /vmlinuz-2.6.18-128.1.1.el5.028stab062.3 ro root=/dev/VolGroupSys/LogVolRoot crashkernel=128M@16M nousb noapic debug=2
and
Code:

options forcedeth optimization_mode=1
Code:

# dmesg |grep forcedeth
forcedeth.c: Reverse Engineered nForce ethernet driver. Version 0.62-Driver Package V1.25.
forcedeth: using HIGHDMA
eth0: forcedeth.c: subsystem: 01043:82b3 bound to 0000:00:0a.0


whig 06-24-09 03:46 AM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
Earlier mentioned,
Code:

service network restart
resumes network; post the requested dump because it could be indicative.

paix 07-06-09 02:42 AM

Re: CentOS 5.2 network bugs, kernel panic (forcedeth, MCP67)
 
In continuation of the subject:
http://www.centos.org/modules/newbb/...topic_id=20835 (CentOS 5.3 nvidia nForce network bugs, kernel panics (forcedeth, MCP67))


All times are GMT -5. The time now is 11:46 PM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.