nV News Forums

 
 

nV News Forums (http://www.nvnews.net/vbulletin/index.php)
-   General Linux (http://www.nvnews.net/vbulletin/forumdisplay.php?f=27)
-   -   What is the NVRAID stripe size? (http://www.nvnews.net/vbulletin/showthread.php?t=78539)

fhj52 10-19-06 03:55 PM

What is the NVRAID stripe size?
 
Hi:

I searched here but found no answer. The NVRAID BIOS does not show detailed parameters about the RAID array.
Q1: What is the stripe size that Media Shield uses(i.e., for ''optimal'' ..., is it always 64k?)

I have looked for some document that explains the suggested method for setup to allow cross-platform usage(MS OS & Linux) as well as just basic steps but have not found such.
Q2: Is there such a doc from Nvidia?

I suppose the most outstanding issue of the Media Shield has been that it is oriented towards benefitting MS OS users because the utilities only run in MS OS.
Q3: Has anything been done to make them Linux friendly? ( i.e., get emails for failed disk and identify the failed disk.)

Thanks!


---------
[Edit]
That's it but I do have some more questions if Q2 answer is no. These are the (more) immediate ones.
For NVRAID:
is it possible to change the stripe size for setup of the array?

is it possible to migrate from 0+1 to another RAID or vice-versa? (Linux or MS OS)

I.e., I need input. ...looking for proper setup & formatting of disks.
E.g., ... is low-level format required if migrating from/to another type of RAID?
Do the raw/new disks need to be partitioned before NVRAID can use them?
Should the partitions be formatted before the RAID array is setup or is the NVRAID going to undo that anyway?

What filesystem, if any specific one, should be used to set the partition type and for formatting before NVRAID is used to set the RAID array type? I.e., does it need to be the same as what will be used on the array or does it matter?

I ask because I have some odd, ithink, output from fdisk and do not know if it is my config and/or setup or something else.
These are (4) 80GB SATA disks that were individually partitioned and formatted before using NVRAID 0+1 BIOS setup. Currently, after using dmraid and having active array, fdisk has some, apparently, unreal info.
fdisk -l -u:
Code:

Disk /dev/sdc: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes

  Device Boot      Start        End      Blocks  Id  System
/dev/sdc1  *          63  310006304  155003121  83  Linux

Disk /dev/sdd: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes

Disk /dev/sde doesn't contain a valid partition table

Disk /dev/sdf: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes

  Device Boot      Start        End      Blocks  Id  System
/dev/sdf1  *          63  310006304  155003121  83  Linux

I don't know why it shows that sdd & sde don't have valid partitons but sdc and sdf do and that troubles me.
The NVRAID array is formatted with one partition as 158723.23MB( 310006305 sectors) and ~1324.28MB of ''free space''(unpartitioned) according to cfdisk. The fdisk -l does not list it at all.
... short version is that, at least, fdisk s/b saying none of the (4) devices have valid partition tables. Question is, is it fdisk or me/NVRAID-setup?

Then dmraid was used to find and activate the RAID. Currently:
#> dmraid -s -s -ccc
Code:

nvidia_bdagicca:312602880:128:raid10:ok:2:4:0
nvidia_bdagicca-0:312602880:128:stripe:ok:0:2:0
/dev/sdf:nvidia:nvidia_bdagicca-0:stripe:ok:156301486:0
/dev/sdd:nvidia:nvidia_bdagicca-0:stripe:ok:156301486:0
nvidia_bdagicca-1:312602880:128:stripe:ok:0:2:0
/dev/sdc:nvidia:nvidia_bdagicca-1:stripe:ok:156301486:0
/dev/sde:nvidia:nvidia_bdagicca-1:stripe:ok:156301486:0

I don't know why d-f and c-e are together rather than c-d and e-f or if makes any difference.

The /dev/mapper/nvidia_bdagicca was partitioned with cfdisk; rebooted and then formatted.
I have been running some filesystem tests so the filesystem has been changed a few times. Hence why I ask for proper procedures.

Although I don't have specific issues/problems at this time with data loss, etc., I surely do not want any later on when the fs is nearly full of data. ;)

If Media Shield RAID utilities are still MS OS bound, what Linux methods are available to get similar functions?

fhj52 10-20-06 05:29 PM

Re: What is the NVRAID stripe size?
 
Hi:

I (re)found the NVRAID users guide but v1.1 is for Microsoft. There is no help in it for Linux. I have not found a version that has Linux info in it. So, Help, please!

There are READ issues with large files and I cannot determine what/why unless I can get some input from documentation or something else.

Specifically, when READing large files of 4GB or 8GB, I get huge loss of transfer rates(like dropping from > 100MB/s to 20-25MB/s) for record sizes that are equal to or larger than 128kB. (using self-compiled iozone to test fs)
It appears to be independent of filesystem as both ext3 and JFS exhibit the same problem.

It could be NVRAID ...or the setup/config ...or something else. I need to determine what.

fhj52 10-22-06 06:42 AM

Re: What is the NVRAID stripe size?
 
I ran some more tests and the huge dropoffs do not occur with other non-raided disks. Tested ATA and SCSI.
I know(well, suspect...) that the performance starts decreasing around 2GB because that is the size of the memory available to each CPU and by 4GB or more, the I/O is going to be the raw disk and fs speed.
However these addtional very large drops do not correlate to anything that I can find except the NVRAID. The disks are capable of up to 150MB/s. With no cache available they won't get that high, of course, but 20-25MB/s is much, much, MUCH too low for READ.

Does the controller cause it?
How can I fix it so that record sizes >128kB won't have that effect with large files?

fhj52 10-22-06 10:05 PM

Re: What is the NVRAID stripe size?
 
So this is what I am talking about:
Code:

iozone -a -i0 -i1 -s 8G -r 128 -r 256 -r512 -Rb /tmp/iozone-NVRAID_JFS=8Gfile_test.xls

        Run began: Sun Oct 22 18:46:49 2006

              KB  reclen  write rewrite    read    reread   
        8388608    128  134176  142128    27766    95556
        8388608    256  138216  143879    28421  105634
        8388608    512  136698  133830    29007  110574

iozone test complete.

Prior tests all show the same thing: for 8GB file, the bottom drops out of the READ as soon as record size hits 128kB. The reread has never been more than 10% higher and, typically, for large files is about the same as the READ since there is not much cache to use.
Smaller record sizes READ at > 112MB/s for the 8GB file. Also, smaller files don't have the dropoff AND it appears to be only for the READ.
Here's a snippet of the data from the full test showing just the READ:
( The top row is records sizes, the left column is file sizes )
Code:

        4        8        16        32        64        128        256        512        1024        2048        4096        8192        16384
1048576        1795491        1857415        2328637        2141122        2387256        2429762        1690177        1383961        988808        1111140        1027648        1115923        911462
2097152        1960697        1960180        1885432        1864134        2303706        2265836        2247662        1547866        1195035        1190274        1142225        1126850        1116831
4194304        135713        215898        201045        193421        165662        148444        188800        96383        70322        117336        119855        155411        132248
8388608        115405        115447        115406        115429        115366        26721        27894        28764        28264        28563        28467        28552        28344


Same test on a SCSI disk(no RAID) shows no big hit:
Code:

iozone -a -i0 -i1 -s 8G -r 128 -r 256 -r512 -Rb /tmp/iozone-SCSI_15k_ext3=8Gfile_test.xls

        Run began: Sun Oct 22 14:15:33 2006

              KB  reclen  write rewrite    read    reread
        8388608    128  62827  52685    46054    59147
        8388608    256  64989  62218    49573    58292
        8388608    512  64537  49897    52131    57109

iozone test complete.

Smaller and larger records all maintained about the same I/O.

Although, oddly, WRITE still outruns READ on all the tests, the results are pointing directly at NVRAID as the cause of the problem.

It appears to be NVRAID controller but there are other things that could be the cause. Tests on the ATA drive are running ...

Edit: IT APPEARS to have the SAME problem as the 128kB record READ is 23MB/s. That's a clear indication to me that the controller is the problem with, of course, possible contributing factors.

ATA drive exhibits the same problem, although the difference between normal and the drop is not as drastic.
Code:

        Run began: Sun Oct 22 21:51:28 2006

              KB  reclen  write rewrite    read    reread   
        8388608    128  44676  36530    24279    40410
        8388608    256  43235  36132    26239    41244
        8388608    512  44912  33881    27399    38586

I need input to determine what causes this.


And is there anybody out there running NVRAID that can verify the same anomaly? Who knows, maybe it is the controller on this mobo ...

Thanks!

fhj52 10-23-06 01:04 PM

Re: What is the NVRAID stripe size?
 
I ran the test on 16GB file last night.
iozone -az -i0 -i1 -S 2048 -s 16G -Rb /tmp/iozone-NVRAID_JFS=16Gfile_test.xls

SAME ISSUE!!
snippet:
( As before, the top row is records sizes(kB), the left column is file size(kB) & data is kB/s. )
Code:

              KB  reclen  write rewrite    read    reread
        16777216      64  120538  124312  113072  113078
        16777216    128  120785  124258    26397    86069
        16777216    256  120480  124435    27605    98806
        16777216    512  119004  123088    28583    105943
        16777216    1024  121796  119224    28473    105762
        16777216    2048  118820  124570    28094    105703
        16777216    4096  119912  123650    28241    105788
        16777216    8192  120574  115020    28222    105759
        16777216  16384  120712  120662    28378    105818

So what is going on here to cause that?
Since this is same problem with ATA it must not be stripe size or anything related to NVRAID directly.
And since the SCSI controller does not exhibit such huge dropoff, it appears that the NVIDIA nv2200 controller is the problem.

WHY is it apparently failing?


Thanks!


All times are GMT -5. The time now is 01:02 AM.

Powered by vBulletin® Version 3.7.1
Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Copyright 1998 - 2014, nV News.