configuring jumbo frames on the SUN SPARC Enterprise T5240 nxge interface

It will be worth to provide some background about jumbo frames.
The maximum transmission unit (MTU) is the size (in bytes) of the largest protocol data unit that it can pass onwards. MTU parameters usually appear in association with a communications interface (NIC, serial port, etc.). The MTU may be fixed by standards (as is the case with Ethernet) or decided at connect time (as is usually the case with point-to-point serial links).
A higher MTU brings greater efficiency because each packet carries more user data while protocol overheads, such as headers or underlying per-packet delays remain fixed, and higher efficiency means a slight improvement in bulk protocol throughput. However, large packets can occupy a slow link for some time, causing greater delays to following packets and increasing lag and minimum latency. For example, a 1500 byte packet, the largest allowed by Ethernet at the network layer (and hence most of the Internet), would tie up a 14.4k modem for about one second.
More can be read on wikipedia.
we need to change on the config file of the NIC to bring changes into effect.
/platform/sun4v/kernel/drv/nxge.conf
# To enable jumbo support for all nxge interfaces,
accept_jumbo = 1;
# To disable jumbo support for all nxge interfaces,
# accept_jumbo = 0;
Also make changes in /etc/hostname.nxge* file like below
root@river03s# cat /etc/hostname.nxge0
river03s  mtu 9000
And then reboot the system. Here are the effects:
After change:
root@river03s# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
zone river03s-z2
inet 127.0.0.1 netmask ff000000
lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
zone river03s-z3
inet 127.0.0.1 netmask ff000000
nxge0: flags=1001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,FIXEDMTU> mtu 9000 index 2
inet xxx.xxx.xxx.xxx netmask ffffe000 broadcast 192.xxx.xx.255
ether 0:21:28:b1:f0:88
nxge1: flags=1001000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4,FIXEDMTU> mtu 9000 index 3
inet xxx232.142.187 netmask ffffff80 broadcast xxx.xxx.142.255
ether 0:21:28:b1:f0:89
nxge1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9194 index 3
zone river03s-z2
inet xxx232.142.193 netmask ffffff80 broadcast xxx232.142.255
nxge1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 9194 index 3
zone river03s-z3
inet xxx.xxx.142.198 netmask ffffff80 broadcast xxx.xxx.142.255
root@river03s# dladm show-link
nxge0           type: non-vlan  mtu: 9194       device: nxge0
nxge1           type: non-vlan  mtu: 9194       device: nxge1
nxge2           type: non-vlan  mtu: 9194       device: nxge2
nxge3           type: non-vlan  mtu: 9194       device: nxge3
Before change:
root@river03s# ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
lo0:1: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
zone river03s-z2
inet 127.0.0.1 netmask ff000000
lo0:2: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
zone river03s-z3
inet 127.0.0.1 netmask ff000000
nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.86.177 netmask ffffe000 broadcast 192.168.95.255
ether 0:21:28:b1:f0:88
nxge1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
inet xxx.xxx.142.187 netmask ffffff80 broadcast xxx.xxx.142.255
ether 0:21:28:b1:f0:89
nxge1:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
zone river03s-z2
inet xxx.xxx.142.193 netmask ffffff80 broadcast xxx.xxx.142.255
nxge1:2: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
zone river03s-z3
inet xxx.xxx.142.198 netmask ffffff80 broadcast xxx.xxx.142.255
root@river03s# dladm show-link
nxge0           type: non-vlan  mtu: 1500       device: nxge0
nxge1           type: non-vlan  mtu: 1500       device: nxge1
nxge2           type: non-vlan  mtu: 1500       device: nxge2
nxge3           type: non-vlan  mtu: 1500       device: nxge3

Comments

Post a Comment

Popular posts from this blog

BMCs and the IPMI Protocol

Image
Baseboard Management Controllers (BMCs) are a type of embedded computer used to provide out-of-band monitoring for desktops and servers. These products are sold under many brand names, including HP iLO, Dell DRAC, Sun ILOM, Fujitsu iRMC, IBM IMM, and Supermicro IPMI. BMCs are often implemented as embedded ARM systems, running Linux and connected directly to the southbridge of the host system's motherboard. Network access is obtained either via 'sideband' access to an existing network card or through a dedicated interface. In addition to being built-in to various motherboards, BMCs are also sold as pluggable modules and PCI cards. Nearly all servers and workstations ship with or support some form of BMC. The Intelligent Platform Management Interface (IPMI) is a collection of specifications that define communication protocols for talking both across a local bus as well as the network. This specification is managed by Intel and currently comes in two flavors, ve...
Read more

Logical Domains Reference Manual

Image
Name ldm– command-line interface for the Logical Domains Manager Synopsis ldm or ldm --help [ subcommand ] ldm -V ldm add-domain -i file ldm add-domain [mac-addr= num ] [hostid= num ] [failure-policy=ignore|panic|reset|stop] [master= master-ldom1 ,..., master-ldom4 ] ldom ldm add-domain ldom ... ldm set-domain -i file ldm set-domain [failure-policy=ignore|panic|reset|stop] [master=[ master-ldom1 ,..., master-ldom4 ]] ldom ldm remove-domain -a ldm remove-domain ldom ... ldm list-domain [-e] [-l] [-o format ] [-p] [ ldom ...] ldm migrate-domain [-n] source-ldom [ user @] target-host [: target-ldom ] ldm add-vcpu number ldom ldm set-vcpu number ldom ldm remove-vcpu number ldom ldm add-crypto number ldom ldm set-crypto number ldom ldm remove-crypto number ldom ldm add-memory size [ unit ] ldom ldm set-memory size [ unit ] ldom ldm remove-memory size [ unit ] ldom ldm cancel-operation (migration | reconf) ldom ldm add-io [bypass=on] bus ...
Read more

Understanding How ZFS Calculates Used Space

This document explains how ZFS calculates the amount of used space within a zpool and provides examples of how various ZFS features affect that calculation. DETAILS A ZFS storage pool is a logical collection of devices that provide space for datasets such as filesystems, snapshots and volumes. A zfs pool can be used as a filesystem, i.e. such as mounting/unmounting; to take snapshots that provides read-only (clones are writable copies of snapshots) copies of the filesystem taken in the past; to create volumes that can be accessed as a raw and a block device and for setting properties on datasets. Interaction of all these datasets and associated properties such as quotas, reservations, compression and deduplication (Solaris 11 only) play a role in how ZFS calculates space usage. Neither the du nor df command have been updated to account for ZFS file system space usage. When calculating zfs space usage, always use the following command. # zfs list -r -o space <pool_name...
Read more