Vic/notes/iscsi/nagle

From Summerseas

Jump to: navigation, search

Contents

[edit]

Server/OS

  • x4200 x86 system with 2 e1000g GigE nics configured for iscsi. Discovery was set to static.
  • SunOS sunx4200-shu01 5.10 Generic_120012-14 i86pc i386 i86pc
[edit]

dd Testing With Default Settings in iscsi.conf

[edit]

Large Block, 64k

  • dd command
    • dd if=/dev/zero of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=64k count=5000
  • iostat command
    • iostat -xn 1 |grep c6t60A98000433465644B4A464A487A3847d0
  • iostat output sample during dd command 
    0.0  728.1    0.0 46595.9  0.0  0.9    0.0    1.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  621.9    0.0 39804.3  0.0  0.9    0.0    1.5   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  691.1    0.0 44227.8  0.0  0.9    0.0    1.4   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  708.0    0.0 45311.2  0.0  0.9    0.0    1.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  653.0    0.0 41792.9  0.0  0.9    0.0    1.4   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  689.0    0.0 44095.0  0.0  0.9    0.0    1.4   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  595.0    0.0 38017.1  0.0  0.8    0.0    1.4   1  83 c6t60A98000433465644B4A464A487A3847d0
[edit]

Small Block, 1k

  • dd command
    • dd if=/dev/zero of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=1k count=50000
  • iostat command
    • iostat -xn 1 |grep c6t60A98000433465644B4A464A487A3847d0
  • iostat output sample during dd command 
    0.0 3622.5    0.0 3622.5  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3833.9    0.0 3833.9  0.0  0.9    0.0    0.2   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3830.1    0.0 3830.1  0.0  0.9    0.0    0.2   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3904.0    0.0 3904.0  0.0  0.9    0.0    0.2   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3720.0    0.0 3720.0  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3658.0    0.0 3658.0  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0


[edit]

dd Testing With tcp-nodelay=1 in iscsi.conf

[edit]

Large Block, 64k

  • dd command
    • dd if=/dev/zero of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=64k count=5000
  • iostat command
    • iostat -xn 1 |grep c6t60A98000433465644B4A464A487A3847d0
  • iostat output sample during dd command 
    0.0  706.0    0.0 45185.1  0.0  0.9    0.0    1.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  718.0    0.0 45954.5  0.0  0.9    0.0    1.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0  620.0    0.0 39676.9  0.0  0.9    0.0    1.5   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  512.0    0.0 32707.0  0.0  0.7    0.0    1.4   0  70 c6t60A98000433465644B4A464A487A3847d0
[edit]

Small Block, 1k

  • dd command
    • dd if=/dev/zero of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=1k count=50000
  • iostat command
    • iostat -xn 1 |grep c6t60A98000433465644B4A464A487A3847d0
  • iostat output sample during dd command 
    0.0 4163.0    0.0 4163.0  0.0  0.9    0.0    0.2   1  93 c6t60A98000433465644B4A464A487A3847d0
    0.0 3923.9    0.0 3923.9  0.0  0.9    0.0    0.2   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 4120.2    0.0 4120.2  0.0  0.9    0.0    0.2   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3164.9    0.0 3164.9  0.0  0.9    0.0    0.3   1  95 c6t60A98000433465644B4A464A487A3847d0



[edit]

dt Testing With Default Settings in iscsi.conf

[edit]

Large Block, 64k

  • dt command
    • dt of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=64k limit=3g iotype=random disable=compare runtime=60 
iostat
    0.0  746.0    0.0 47744.1  0.0  1.0    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  690.0    0.0 44160.3  0.0  1.0    0.0    1.4   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  730.0    0.0 46719.5  0.0  1.0    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  745.0    0.0 47682.0  0.0  1.0    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  551.0    0.0 35264.2  0.0  1.0    0.0    1.7   0  96 c6t60A98000433465644B4A464A487A3847d0
    0.0  667.0    0.0 42686.9  0.0  1.0    0.0    1.4   1  96 c6t60A98000433465644B4A464A487A3847d0


[edit]

Small Block, 1k

  • dt command
    • dt of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=1k limit=3g iotype=random disable=compare runtime=60 
iostat
    0.0 3493.9    0.0 3493.9  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3528.0    0.0 3528.0  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3529.1    0.0 3529.1  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3503.6    0.0 3503.6  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
[edit]

dt Testing With tcp-nodelay=1 in iscsi.conf

[edit]

Large Block, 64k

  • dt command
    • dt of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=64k limit=3g iotype=random disable=compare runtime=60 

iostat
    0.0  751.1    0.0 48067.4  0.0  0.9    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  652.1    0.0 41732.8  0.0  1.0    0.0    1.5   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  754.8    0.0 48309.4  0.0  0.9    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  706.1    0.0 45188.1  0.0  1.0    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0
    0.0  719.9    0.0 46072.0  0.0  1.0    0.0    1.3   1  95 c6t60A98000433465644B4A464A487A3847d0


[edit]

Small Block, 1k

  • dt command
    • dt of=/dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s6 bs=1k limit=3g iotype=random disable=compare runtime=60 
iostat
    0.0 3550.9    0.0 3550.9  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3537.2    0.0 3537.2  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3549.6    0.0 3549.6  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0
    0.0 3454.1    0.0 3454.1  0.0  0.9    0.0    0.3   1  94 c6t60A98000433465644B4A464A487A3847d0


[edit]

Path info for iscsi disk device under test...

[root@sunx4200-shu01--->]mpathadm show lu /dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s2
Logical Unit:  /dev/rdsk/c6t60A98000433465644B4A464A487A3847d0s2
        mpath-support:  libmpscsi_vhci.so
        Vendor:  NETAPP
        Product:  LUN
        Revision:  0.2
        Name Type:  unknown type
        Name:  60a98000433465644b4a464a487a3847
        Asymmetric:  no
        Current Load Balance:  round-robin
        Logical Unit Group ID:  NA
        Auto Failback:  on
        Auto Probing:  NA

        Paths:
                Initiator Port Name:  iqn.1986-03.com.sun:01:fa68e9efffff.4762e221,4000002a00ff
                Target Port Name:  4000002a0000,iqn.1992-08.com.netapp:sn.101177625,1001
                Override Path:  NA
                Path State:  OK
                Disabled:  no

                Initiator Port Name:  iqn.1986-03.com.sun:01:fa68e9efffff.4762e221,4000002a00ff
                Target Port Name:  4000002a0000,iqn.1992-08.com.netapp:sn.101177625,1002
                Override Path:  NA
                Path State:  OK
                Disabled:  no

        Target Ports:
                Name:  4000002a0000,iqn.1992-08.com.netapp:sn.101177625,1001
                Relative ID:  0

                Name:  4000002a0000,iqn.1992-08.com.netapp:sn.101177625,1002
                Relative ID:  0
[edit]

SIO Testing on UFS with forcedirectio

[edit]

SIO RESULTS With Default Settings

Test Block Size IOPS TPUT
Sequential Read 512 40052 20026
Sequential Read 4096 29466 117864
Sequential Read 8192 27488 219906
Sequential Read 65536 3586 229512
Sequential Write 512 24015 12008
Sequential Write 4096 14102 56409
Sequential Write 8192 9886 79090
Sequential Write 65536 1790 114561
Random read 512 24091 12046
Random read 4096 35286 141144
Random read 8192 27314 218512
Random read 65536 3581 229156
Random Write 512 14578 7289
Random Write 4096 12093 48373
Random Write 8192 8451 67604
Random Write 65536 1613 103244


[edit]

SIO RESULTS With tcp-nodelay=1

Test Block Size IOPS TPUT
Sequential Read 512 37152 18576
Sequential Read 4096 32129 128517
Sequential Read 8192 28397 227174
Sequential Read 65536 3580 229136
Sequential Write 512 23737 11869
Sequential Write 4096 14260 57041
Sequential Write 8192 9878 79026
Sequential Write 65536 1787 114357
Random read 512 26686 13343
Random read 4096 36964 147856
Random read 8192 28241 225925
Random read 65536 3586 229508
Random Write 512 14270 7135
Random Write 4096 12097 48387
Random Write 8192 8479 67831
Random Write 65536 1616 103437
[edit]

COMMANDS

SIO Command
/usr/local/test/bin/sio 100 0 512 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 0 4096 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 0 8192 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 0 65536 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 0 512 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 0 4096 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 0 8192 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 0 65536 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 100 512 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 100 4096 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 100 8192 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 100 100 65536 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 100 512 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 100 4096 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 100 8192 0 1g 140 32 /vol1/perftest
/usr/local/test/bin/sio 0 100 65536 0 1g 140 32 /vol1/perftest
Retrieved from "http://www.vicengle.org/wiki/Vic/notes/iscsi/nagle"
Personal tools