http://www.vilett.com/disParity/forum/ disParity: Recent Topics en Sat, 19 May 2012 03:28:42 +0000 http://www.vilett.com/disParity/forum/topic.php?id=148#post-830 Wed, 08 Feb 2012 10:14:37 +0000 Roland 830@http://www.vilett.com/disParity/forum/ <p>I recently upgraded my media server to a 64 bit OS, and also increased the RAM to 4GB (the most the aging mobo will support.)</p> <p>An area where disParity can benefit from 64 bits and the extra virtual memory available is the temp file it generates while processing a file. In order to minimize the risk of parity corruption due to an unexpected failure, parity changes due to an add or delete are written to a temp file first, and then flushed to the parity set after the add or delete is finished. For small files (typically less than 1GB) disParity currently uses a memory buffer for this temp file, but for larger files, it has no choice but to write it to disk, and then read it back from disk and write it to parity, which significantly slows down parity updates for large files.</p> <p>I've tinkered with the code a bit, and tried switching the "memory buffer" implementation to one using memory mapped files instead. It seems to provide a dramatic improvement to parity updates for the cases where it would have previously used an on-disk temp file.</p> <p>Is this something any current disParity users on 64 bit out there would find useful? There are a couple of drawbacks I know of:</p> <p>1) The change requires updating to version 4 of the .NET runtime, which is where MS added support for memory mapped files. Not a huge deal, but I liked the fact that previously disPariity only required version 2, which just about everyone has these days.</p> <p>2) It's hard for disParity to know how big a memory mapped file it can get away with. This is a function of the amount of RAM in the machine and how much of it is currently available for the file. If disParity tries to create a memory mapped file that is too large, it will work, but system performance slows to an absolute crawl. Ultimately, the "max" size for the temp file is something that the user will have to tune, which is not very user friendly, unless I can come up with some other way to figure this out automatically.</p> <p>-Roland </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-823 Fri, 06 Jan 2012 13:02:25 +0000 Phatty2x4 823@http://www.vilett.com/disParity/forum/ <p>anyone had a chance to play with these yet? </p> http://www.vilett.com/disParity/forum/topic.php?id=59#post-287 Sat, 26 Sep 2009 05:57:55 +0000 Neostim 287@http://www.vilett.com/disParity/forum/ <p>Hello there,</p> <p>I came accross this software yesterday (after my RAID5 died :( ) and I'm really liking the concept, my only problem is I'm used to having one logical drive storing TB's of data, and I'd rather not have to maintain 4 separate TB hard drives distributing files among them.</p> <p>Is there any way of grouping them together to appear as one drive? Creating any type of volume in windows wouldn't work with disParity I don't think, as disParity needs the drive letters to work off of.</p> <p>If there is any method of software you could tell me about, that would be GREAT!</p> <p>Thanks! </p> http://www.vilett.com/disParity/forum/topic.php?id=103#post-700 Thu, 09 Sep 2010 21:01:03 +0000 Killroy 700@http://www.vilett.com/disParity/forum/ <p>Is there any way to add a feature that can allow disParity to skip a size that is smaller than a certain size or certain extensions? As a media server I have a ton of .jpgs, .xml, .NFO., etc, that are very small and are constantly updated by my media catalog software. I really don't care if I lose these files if I have to perform a recovery as I can easily rebuild them with the software if I re-scan my media but it can really fowl up the parity with constant updates to such small files where it creates a fragmentation to the parity.</p> <p>Thanks. </p> http://www.vilett.com/disParity/forum/topic.php?id=145#post-815 Tue, 24 May 2011 05:45:12 +0000 Phatty2x4 815@http://www.vilett.com/disParity/forum/ <p>I use Windows XP as my NAS. I have a few TB of data.</p> <p>Normally, when I create a new parity set, it will take roughly 7 to 8 hours.</p> <p>I did a lot of house work on the nas and decided this would be a great time to recreate the whole protection parity from scratch.</p> <p>I still have the same amount of data (give or take just a few 100 gigs). This time it took close to 36 hours to complete.</p> <p>Nothing has changed on my systems from the last time I created the parity except for new system patches.</p> <p>Has anyone run in to this? </p> http://www.vilett.com/disParity/forum/topic.php?id=146#post-817 Tue, 31 May 2011 10:16:35 +0000 chiggyman 817@http://www.vilett.com/disParity/forum/ <p>I've been using DisParity for some time now, but recently due to hard drive re arragements i've decided to include a drive that's more frequently updated in the parity. During the original parity creation (of 7TB ish) it's hard to ensure this drive doesn't have any changes.<br /> If the drive does have any changes while the parity is being written, will this cause problems upon recovery? Or will an update command after the initial creation bring it back in line? </p> http://www.vilett.com/disParity/forum/topic.php?id=144#post-812 Sat, 14 May 2011 13:16:10 +0000 tahngarth 812@http://www.vilett.com/disParity/forum/ <p>I lost a hard drive, one partition had data protected by disparity, the other partition had windows, where the config file was at. I have no recent back up of the windows partition (sigh, stupid me), so is there any way to figure out the config based on the parity data? </p> http://www.vilett.com/disParity/forum/topic.php?id=57#post-269 Sun, 13 Sep 2009 21:24:54 +0000 Cool1Net6 269@http://www.vilett.com/disParity/forum/ <p>About a month ago I stumbled across this application when trying to find out more information on FlexRAID. What intrigued me about it is that it works exactly like FlexRAID, but I could actually understand how it works and use it quite easily. Scanning this forum, I also discovered that a GUI was being requested, and Roland himself was unavailable for the job. Tada! I see a weekend project!</p> <p>After heavily coding for one weekend (hours on end), I was able to get a functioning prototype GUI that contained the three core functions (create, update, restore) and gave information on the drives status. Since then I updated the app to automate all of disParity's functions and a few optimizations here and there, and a month later (to the day) I want to present what I have developed so far.</p> <p>I call it the "DisParity Storage Manager". Although this is a working copy, it hasn't met my quality bar and I do not want to release it, however, here are a few screen shots of progress.</p> <p><a href="http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Main.png" rel="nofollow">http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Main.png</a></p> <p><a href="http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Menu.png" rel="nofollow">http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Menu.png</a></p> <p><a href="http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Building.png" rel="nofollow">http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Building.png</a></p> <p><a href="http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Finished.png" rel="nofollow">http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Finished.png</a></p> <p><a href="http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Remove.png" rel="nofollow">http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Remove.png</a></p> <p><a href="http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Broken.png" rel="nofollow">http://i64.photobucket.com/albums/h176/StepForwarD/DSP_Broken.png</a></p> <p>Its going to take me a while to really perfect this application as work just got more intense and I have less time to focus my effort here, but if you like what you see or if anyone is even still interested in the development of a GUI, just reply to this thread. I will check on it every now and then.</p> <p>Thanks.</p> <p>-Cool- </p> http://www.vilett.com/disParity/forum/topic.php?id=135#post-799 Tue, 15 Mar 2011 13:07:11 +0000 Duff26 799@http://www.vilett.com/disParity/forum/ <p>I just got rid of my software raid but now some disks are basic and some are dynamic. I cannot convert back to basic unless I format the drives which I would prefer not to. Does disParity have any problem with the differences in drive configurations like that? Also I understand that there is a parity disk and the parity is stored there, but what if multiple drives change data and disParity has not scanned and one of the disks fails. Will the disk be brought back to the previous state when scanned or be corrupt because the parity calculated does not match the new data on some of the other drives?</p> <p>Thanks! </p> http://www.vilett.com/disParity/forum/topic.php?id=134#post-797 Thu, 24 Feb 2011 13:47:48 +0000 airjrdn 797@http://www.vilett.com/disParity/forum/ <p>I've been testing disParity lately and noticed that when I deleted ~25G of data from my largest dataX, the total size of the parity store didn't decrease. Should it have?</p> <p>For example, I have the following config:</p> <p>parity=G:\</p> <p>data1=H:\<br /> data2=I:\<br /> data3=J:\<br /> data4=K:\<br /> data5=L:\<br /> data6=P:\<br /> data7=U:\<br /> data8=D:\Images\</p> <p>G, H, and I are 1TB drives, the rest are smaller. H had the most data, with only a couple of GBs free. G probably had 30G or so free.</p> <p>I ran a create, and a day and a half or so later, it was done (G is an external USB drive, so that was probably the bottleneck). The amount of free space on G was pretty close to the same as H.</p> <p>After that, I deleted 25G or so from H and ran an update. I assumed the free space on G would increase, but it didn't. Should it have? </p> http://www.vilett.com/disParity/forum/topic.php?id=110#post-745 Sat, 01 Jan 2011 09:32:43 +0000 paxtrix 745@http://www.vilett.com/disParity/forum/ <p>What would be my options (RAID 6) to protect against 2 drive failures or loss of parity drive? I'm current adding the last 3 drives to my parity (for some reason it would error whenever I tried to add all 13 data risk units. I thought this procedure, from what I found on the forum here, would expose my possible faulty drive but the process continues... It has taken several days to arrive at this point in the process on my WHS. My thought is to use the 2TB drive that I'm currently using as my temp drive (again great info on the forums as my c:\ temp location wasn't able to process my Blu-rays) to copy all parity data over to this drive and process 2 updates by changing the parity drive assignment in my config file. I would assume this would cover me in the event of a crash of the main 2TB parity drive. Would this allow for the recovery of 2 data risk units if I were to lose 2 drives?</p> <p>Thanks for the help &#38; this is a great simple option to my previous FlexRAID! Nice Roland!!! </p> http://www.vilett.com/disParity/forum/topic.php?id=133#post-791 Wed, 23 Feb 2011 12:06:12 +0000 Duff26 791@http://www.vilett.com/disParity/forum/ <p>Thinking about implementing disParity on my HTPC. I don't care at all about performance gains and the files rarely change but was looking for an option that was able me to store the most amount of data but still be recoverable from a 1 disk failure.</p> <p>My question is what is the efficiency of disParity? is it similar to RAID 5 with a 1 - 1/disks efficiency? and this is very reliable as long as I do not change the drive configuration? </p> http://www.vilett.com/disParity/forum/topic.php?id=130#post-776 Thu, 27 Jan 2011 11:31:47 +0000 Roland 776@http://www.vilett.com/disParity/forum/ <p>Looks like reCAPTCHA has been hacked...again. We got a huge flood of new spam accounts. I've disabled new account registration until they can resolve the issue. If you need an account, send me an email at <a href="mailto:rolandv@gmail.com.">rolandv@gmail.com.</a> </p> http://www.vilett.com/disParity/forum/topic.php?id=107#post-726 Thu, 21 Oct 2010 11:40:18 +0000 gprime 726@http://www.vilett.com/disParity/forum/ <p>Hi, i have not tested disParity yet but i'll do soon. I have one question before testing it. If i setup disParity on my system and i format my windows or i take my drive to an other computer and i reinstall disParity. Does disParity will recognize the parity drive or know where to restore data on the new system (if the drive letter of my data drive have changed ?). Thanks and sorry if my English is not perfect ^^ </p> http://www.vilett.com/disParity/forum/topic.php?id=132#post-779 Wed, 16 Feb 2011 18:48:15 +0000 protovision 779@http://www.vilett.com/disParity/forum/ <p>Hi All,</p> <p>Anyone have a recommendation for a pci-e sata card with minimum 4 internal sata2 connectors?</p> <p>I need to replace my PCI based 4 port sata with something faster for parity generation, and have a pci-e x4 slot available. I've seen a couple of x1 cards (sil-3124 chipset) and x4 cards (Marvell 88SX7042 chipset), anyone know if the x4 will make a big difference with parity generation? (in theory it should, but would like real-world answers)</p> <p>thx! </p> http://www.vilett.com/disParity/forum/topic.php?id=131#post-778 Wed, 16 Feb 2011 18:38:52 +0000 protovision 778@http://www.vilett.com/disParity/forum/ <p>Hey there,</p> <p>anyone know if there's any advantage to 'balancing' the amount of data on the drives before creating the parity file?</p> <p>for example, if I have 700GB(1.5TB drive) and 1.5TB(2TB drive) used, would it be better to spread the data evenly, or will the smaller drive get removed from parity generation once it's 'done', and the parity file will generate quicker for the remaining data on the 2TB? </p> http://www.vilett.com/disParity/forum/topic.php?id=111#post-750 Fri, 07 Jan 2011 16:45:10 +0000 Roland 750@http://www.vilett.com/disParity/forum/ <p>I just ran the "stats" command for the first time in a long time.</p> <p>I got just over 33,000 files, 8.37 TB of data total.</p> <p>How much data are other people protecting with disParity? </p> http://www.vilett.com/disParity/forum/topic.php?id=114#post-759 Wed, 26 Jan 2011 18:51:12 +0000 havnfn 759@http://www.vilett.com/disParity/forum/ <p>Is it possible to protect drives that are not connected to the computer all the time, as in a external drive? </p> http://www.vilett.com/disParity/forum/topic.php?id=109#post-744 Tue, 28 Dec 2010 13:33:34 +0000 Neostim 744@http://www.vilett.com/disParity/forum/ <p>I haven't tried this yet, but in theory I'm thinking it should work.</p> <p>If I encrypt all of my drives using TrueCrypt, and then mount them as volumes in windows (which would show up as drives), I should be able to run disparity across the mounted drives (once they have been unencrypted with a password).</p> <p>Has anyone thought of or tried this, any drawbacks? </p> http://www.vilett.com/disParity/forum/topic.php?id=108#post-736 Wed, 01 Dec 2010 19:17:52 +0000 axeman48 736@http://www.vilett.com/disParity/forum/ <p>Hi Roland, thanks for the awesome software. I have a problem, disParity keeps crashing at random points into the initial creation. Here is a snip from the log:</p> <p>Fatal error encountered during parity generation: Data error (cyclic redundancy check).</p> <p>Stack trace: at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)<br /> at System.IO.FileStream.ReadCore(Byte[] buffer, Int32 offset, Int32 count)<br /> at System.IO.FileStream.Read(Byte[] array, Int32 offset, Int32 count)<br /> at Disparity.DataPath.ReadData(Byte[] buf, UInt32 block)<br /> at Disparity.Program.Create()</p> <p>I have checked the drives to be clean, memory to be good. Using Win 7 X64. Please let me know if you have any ideas or if you need any additional information. Thanks! </p>