Vdc Driver - Configuring Vdisk Physical Block Size in vdc.conf on a Guest Ldom

Goal
This document details a new feature in the lastest version of Solaris 11.1.11.4. helping with LUN alignement on NetApp filer using 4k sectors with a OVM for SPARC Guest Domain. (LDOM).


Previously one needed to implement settings described in document 1585893.1 in order to get ZFS pool ashift to equal 12. These settings don't work for a Guest Domain using Virtual Disk Client driver (vdc driver) as the check done on the disks to control the physical block size are done directly by the disk target driver vdc not the Solaris disk target driver sd/ssd.

When disk are controlled by vdc driver on a guest ldom, you can see following device path in format output:

format:

AVAILABLE DISK SELECTIONS:
       0. c0d0 <SUN-ZFSStorage7420-1.0 cyl 973 alt 2 hd 254 sec 254>
          /virtual-devices@100/channel-devices@200/disk@0
       1. c0d1 <SUN-ZFSStorage7420-1.0 cyl 973 alt 2 hd 254 sec 254>
          /virtual-devices@100/channel-devices@200/disk@1
       2. c0d2 <SUN-ZFS Storage 7420-1.0-260.00GB>
          /virtual-devices@100/channel-devices@200/disk@2Another way to check this is iostat -iE output, the Product field will shows VDSK:


iostat -iE:

vdc0      Soft Errors: 9 Transport Errors: 67 Protocol Errors: 0
Vendor: SUN Product: VDSK Size: 32.21GB <32212254720 bytes>

vdc1      Soft Errors: 4 Transport Errors: 56 Protocol Errors: 0
Vendor: SUN Product: VDSK Size: 32.21GB <32212254720 bytes>

vdc2      Soft Errors: 0 Transport Errors: 0 Protocol Errors: 0
Vendor: SUN Product: VDSK Size: 279.17GB <279172874240 bytes>In such configuration if one wanted to maintain the alignement the workaround was to create the zpool on the control domain with the original ssd-config-list entry for NetApp in ssd.conf and export zfs volume dataset to the guest domain.

With the release of Solaris 11.1.11.4 one can now give full access to NetAPP logical unit to a guest ldom and create a zpool which will have an ashift value of 12.
This was made possible by the integration of Bug 15824910: Add support to configure vdisk physical block size in vdc.conf


Solution
1. Create vdisk based on whole NetAPP logical unit and assign them to the guest ldom
2. Configure the physical block size of the vdisk in vdc.conf
/*
       * The physical block size of a virtual disk can be specified in
       * vdc.conf. The syntax is:
       *
       *      block-size-list="<instance>:<pbsize>", ... ;
       *
       * where:
       *      <instance> is the virtual disk instance
       *      <pbsize> is the physical block size
      In /platform/sun4v/kernel/drv/vdc.conf add the following entry to set the block size to 4k for vdc instance 1


block-size-list="1:4096"; 3. Example
Modify vdc.conf

root@sol11-test:~# grep vdc /etc/path_to_inst
"/virtual-devices@100/channel-devices@200/disk@0" 0 "vdc"
"/virtual-devices@100/channel-devices@200/disk@1" 1 "vdc"

root@sol11-test:~# cat /platform/sun4v/kernel/drv/vdc.conf
#
# Copyright (c) 2011, Oracle and/or its affiliates. All rights reserved.
#

#
# Associate the driver with devid resolution.
#
block-size-list="1:4096";Now create the zpool in the guest ldom

root@sol11-test:~# zpool create -f test c2d1
root@sol11-test:~# zdb -L test | grep ashift
              ashift: 12The NetAPP storage do not report anymore misaligned i/o

uc1hdwtlnap001*>  lun alignment show /vol/sol11test_data_vol/sol11test_data
      /vol/sol11test_data_vol/sol11test_data
              Multiprotocol type: solaris
              Alignment: aligned
              Write alignment histogram percentage: 85, 0, 0, 0, 0, 0, 0, 0
              Read alignment histogram percentage: 93, 0, 1, 0, 0, 0, 0, 0
              Partial writes percentage: 14