http://www.vilett.com/disParity/forum/ disParity: Recent Posts en Sat, 19 May 2012 03:28:15 +0000 http://www.vilett.com/disParity/forum/topic.php?id=148#post-834 Mon, 14 May 2012 04:46:12 +0000 Neostim 834@http://www.vilett.com/disParity/forum/ <p>I too would like this feature :) </p> http://www.vilett.com/disParity/forum/topic.php?id=148#post-833 Sun, 22 Apr 2012 15:18:02 +0000 freehand 833@http://www.vilett.com/disParity/forum/ <p>I would find this feature really useful. Going to .NET 4.0 is fine with me. The install is pretty small. </p> http://www.vilett.com/disParity/forum/topic.php?id=148#post-832 Wed, 04 Apr 2012 11:03:57 +0000 Roland 832@http://www.vilett.com/disParity/forum/ <p>I'm still trying to work out how to make this change available without making it too easy for people to accidentally crush their servers with too much VM use.</p> <p>As for memory usage during snapshot creation, I'm not sure how a setting there would help. Snapshot creation (i.e. the "create" command) performance is limited by disk speed. Having more RAM available wouldn't help in any way that I can see. </p> http://www.vilett.com/disParity/forum/topic.php?id=148#post-831 Wed, 04 Apr 2012 09:46:39 +0000 Satsuki 831@http://www.vilett.com/disParity/forum/ <p>i'm realy waiting for it.<br /> as for the 32bits version, it would be cool to be able to set a memory size for disparity to use to improve snapshot creation ^^</p> <p>thanks for this great software </p> http://www.vilett.com/disParity/forum/topic.php?id=148#post-830 Wed, 08 Feb 2012 10:14:37 +0000 Roland 830@http://www.vilett.com/disParity/forum/ <p>I recently upgraded my media server to a 64 bit OS, and also increased the RAM to 4GB (the most the aging mobo will support.)</p> <p>An area where disParity can benefit from 64 bits and the extra virtual memory available is the temp file it generates while processing a file. In order to minimize the risk of parity corruption due to an unexpected failure, parity changes due to an add or delete are written to a temp file first, and then flushed to the parity set after the add or delete is finished. For small files (typically less than 1GB) disParity currently uses a memory buffer for this temp file, but for larger files, it has no choice but to write it to disk, and then read it back from disk and write it to parity, which significantly slows down parity updates for large files.</p> <p>I've tinkered with the code a bit, and tried switching the "memory buffer" implementation to one using memory mapped files instead. It seems to provide a dramatic improvement to parity updates for the cases where it would have previously used an on-disk temp file.</p> <p>Is this something any current disParity users on 64 bit out there would find useful? There are a couple of drawbacks I know of:</p> <p>1) The change requires updating to version 4 of the .NET runtime, which is where MS added support for memory mapped files. Not a huge deal, but I liked the fact that previously disPariity only required version 2, which just about everyone has these days.</p> <p>2) It's hard for disParity to know how big a memory mapped file it can get away with. This is a function of the amount of RAM in the machine and how much of it is currently available for the file. If disParity tries to create a memory mapped file that is too large, it will work, but system performance slows to an absolute crawl. Ultimately, the "max" size for the temp file is something that the user will have to tune, which is not very user friendly, unless I can come up with some other way to figure this out automatically.</p> <p>-Roland </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-829 Sun, 22 Jan 2012 19:40:58 +0000 Roland 829@http://www.vilett.com/disParity/forum/ <p>OK, whew! </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-828 Sat, 21 Jan 2012 20:19:28 +0000 BlkKnight 828@http://www.vilett.com/disParity/forum/ <p>There's no 10TB limit. It's just the logical size chosen for the example. From the MSDN post:</p> <p>"...subject to its maximum logical size of 10TB. If needed, you can certainly also increase the maximum logical size of a space."</p> <p>-BK </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-827 Sat, 21 Jan 2012 09:23:29 +0000 Roland 827@http://www.vilett.com/disParity/forum/ <p>10TB limit? I already have more than that now on my media server.</p> <p>Strange that a next-gen technology like that would have such a modest limit by today's standards. </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-826 Tue, 17 Jan 2012 14:33:29 +0000 PCWiz 826@http://www.vilett.com/disParity/forum/ <p>As much as I would like to play with this new Windows 8 feature (Storage Spaces) it does not look like it was release in the Windows 8 developer preview. But it does not look like we’ll have to wait long to see it and play with it as it is supposed to be part of the Windows 8 public Beta, which is due out late February 2012 as per toms hardware:<br /> <a href="http://www.tomshardware.com/news/Microsoft-Windows-8-Windows-Store-Public-Beta-Metro,14195.html" rel="nofollow">http://www.tomshardware.com/news/Microsoft-Windows-8-Windows-Store-Public-Beta-Metro,14195.html</a></p> <p>Here is a link to the MSDN Blog “An inside look from the Windows engineering team” detailing some of the features and of Storage Spaces.<br /> <a href="http://blogs.msdn.com/b/b8/archive/2012/01/05/virtualizing-storage-for-scale-resiliency-and-efficiency.aspx" rel="nofollow">http://blogs.msdn.com/b/b8/archive/2012/01/05/virtualizing-storage-for-scale-resiliency-and-efficiency.aspx</a></p> <p>This is definitely something on my radar of things to watch for once the Windows 8 Beta is out. </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-825 Wed, 11 Jan 2012 10:15:45 +0000 Roland 825@http://www.vilett.com/disParity/forum/ <p>I just upgraded to Windows 7! :) </p> http://www.vilett.com/disParity/forum/topic.php?id=59&page=2#post-824 Sat, 07 Jan 2012 19:11:54 +0000 rust0r 824@http://www.vilett.com/disParity/forum/ <p>For anyone still looking something like this:</p> <p><a href="http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html" rel="nofollow">http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html</a></p> <p>Link Shell Extension, works great for grouping all of my various TV Show drives into one.</p> <p>Thanks for everyone hard work on other release projects </p> http://www.vilett.com/disParity/forum/topic.php?id=147#post-823 Fri, 06 Jan 2012 13:02:25 +0000 Phatty2x4 823@http://www.vilett.com/disParity/forum/ <p>anyone had a chance to play with these yet? </p> http://www.vilett.com/disParity/forum/topic.php?id=103#post-822 Sat, 03 Sep 2011 17:21:56 +0000 freehand 822@http://www.vilett.com/disParity/forum/ <p>Roland any update on this? I am now facing this problem as well. Some sort of filter to exclude matching files would be great!</p> <p>For now I have made a small program to hide all .jpg, .nfo files before disParity runs and then unhides them after. </p> <p>Something built into disParity would be awesome tho! </p> http://www.vilett.com/disParity/forum/topic.php?id=145#post-821 Thu, 28 Jul 2011 19:34:46 +0000 Phatty2x4 821@http://www.vilett.com/disParity/forum/ <p>Not only obsolete, it's almost not worth the hassle to sell on e-bay. I pretty much just give the drives away anymore. </p> http://www.vilett.com/disParity/forum/topic.php?id=145#post-820 Thu, 28 Jul 2011 09:28:40 +0000 Roland 820@http://www.vilett.com/disParity/forum/ <p>Glad to hear you got it resolved.</p> <p>Personally I've been pulling out my 750GB drives and replacing them with 2TB drives. I can't believe a drive I only bought 3 years ago is already obsolete. </p> http://www.vilett.com/disParity/forum/topic.php?id=145#post-819 Wed, 27 Jul 2011 18:40:45 +0000 Phatty2x4 819@http://www.vilett.com/disParity/forum/ <p>Looks like it was some issue with one of the drives I was using. Individually, I couldn't find any error. Sting the drives together and they would crawl to a halt.</p> <p>To be safe, I upgraded my drives to bigger size and ended up reducing the amount of needed drives by three.</p> <p>Now everything is back to normal. </p> http://www.vilett.com/disParity/forum/topic.php?id=146#post-818 Tue, 31 May 2011 16:25:41 +0000 Roland 818@http://www.vilett.com/disParity/forum/ <p>The biggest risk is from a file changing while disParity is reading it. That will cause unpredictable behavior and almost certainly recovery issues later. </p> <p>If a file changes after disParity has already added it to parity, that should be okay, although a proper recovery won't be possible until a subsequent "update" is run to pick up the changes.</p> <p>However, if a file changes *after* the create/update has begun, but *before* disParity reads it, there could possibly be issues (since the file attributes captured at the start have since changed by the time disParity reads it.) It's hard for me to predict exactly what will happen here, I'd have to stare at code for a while. Suffice it to say, I cannot guarantee that this will work. </p> http://www.vilett.com/disParity/forum/topic.php?id=146#post-817 Tue, 31 May 2011 10:16:35 +0000 chiggyman 817@http://www.vilett.com/disParity/forum/ <p>I've been using DisParity for some time now, but recently due to hard drive re arragements i've decided to include a drive that's more frequently updated in the parity. During the original parity creation (of 7TB ish) it's hard to ensure this drive doesn't have any changes.<br /> If the drive does have any changes while the parity is being written, will this cause problems upon recovery? Or will an update command after the initial creation bring it back in line? </p> http://www.vilett.com/disParity/forum/topic.php?id=145#post-816 Tue, 24 May 2011 10:42:45 +0000 Roland 816@http://www.vilett.com/disParity/forum/ <p>Very strange that the times would change that much. I can't explain it. My only guess is that perhaps one of your drives is developing issues which could be reducing the throughput from that drive, but I admit that's a long shot. </p> http://www.vilett.com/disParity/forum/topic.php?id=145#post-815 Tue, 24 May 2011 05:45:12 +0000 Phatty2x4 815@http://www.vilett.com/disParity/forum/ <p>I use Windows XP as my NAS. I have a few TB of data.</p> <p>Normally, when I create a new parity set, it will take roughly 7 to 8 hours.</p> <p>I did a lot of house work on the nas and decided this would be a great time to recreate the whole protection parity from scratch.</p> <p>I still have the same amount of data (give or take just a few 100 gigs). This time it took close to 36 hours to complete.</p> <p>Nothing has changed on my systems from the last time I created the parity except for new system patches.</p> <p>Has anyone run in to this? </p> http://www.vilett.com/disParity/forum/topic.php?id=144#post-814 Mon, 16 May 2011 12:26:11 +0000 tahngarth 814@http://www.vilett.com/disParity/forum/ <p>Thanks Roland!<br /> I tried that and it mostly worked. I was able to recover 70% of the files, the rest failed the hash check ( I checked some text files, complete garbage ). I think part of the problem is all of a sudden a few files on other drives are being detected as changed. I presume somehow the file got corrupted. I run updates every night, so I'm not sure what happened. </p> <p>Regardless, awesome piece of software. This is now the second time it has saved me time and heartache! Now to find something to protect that pesky windows drive. </p> http://www.vilett.com/disParity/forum/topic.php?id=144#post-813 Mon, 16 May 2011 10:36:45 +0000 Roland 813@http://www.vilett.com/disParity/forum/ <p>Possibly! </p> <p>What you'll need to reconstruct yourself is that the roots were of all the data drives. The parity meta data deliberately does not store the root of the drive (this allows you to do some tricky things like move drives around in your array and avoid having to do any parity recalculation if you just update your config file correctly to represent the new arrangement.)</p> <p>I assume that if you saw what the file names were on each of the data drives, you could figure out what the roots were again, either from inspecting your remaining data drives, or from memory in the case of the one lost partition?</p> <p>You can open the meta data files (files0.dat, files1.dat, etc.) in a binary file editor, and you should be able to see text file names of the files on that drive.</p> <p>So for example if you open files0.dat and you see some .mkv files in there that you know are on the G: drive, then you know data1 must be G:\. The only possibly confusing thing is you have to remember that "data1" in the config.txt corresponds to files0.dat, "data2" is files1.dat, etc.</p> <p>-Roland </p> http://www.vilett.com/disParity/forum/topic.php?id=144#post-812 Sat, 14 May 2011 13:16:10 +0000 tahngarth 812@http://www.vilett.com/disParity/forum/ <p>I lost a hard drive, one partition had data protected by disparity, the other partition had windows, where the config file was at. I have no recent back up of the windows partition (sigh, stupid me), so is there any way to figure out the config based on the parity data? </p> http://www.vilett.com/disParity/forum/topic.php?id=57&page=2#post-811 Wed, 04 May 2011 06:47:49 +0000 protovision 811@http://www.vilett.com/disParity/forum/ <p>thanks for the info Roland!</p> <p>I thought the temp size and folder were used to create/hold the 1GB parity 'chunk', so 1.2GB would be big enough, oh well. </p> http://www.vilett.com/disParity/forum/topic.php?id=57&page=2#post-810 Tue, 03 May 2011 09:40:41 +0000 Roland 810@http://www.vilett.com/disParity/forum/ <p>I don't use DSM...but as for the temp RAM setting, that is only used when the files being processed are smaller than the RAM setting. If they are larger, disk must be used instead for the temp storage.</p> <p>If you are pointing your temp folder to a disk with only 1.5GB free, and then processing files that are 8-20GB each, that will NOT work!!! You should be seeing errors in your disParity output if this is the case. </p> <p>The purpose of the temp stuff is to avoid parity corruption due to errors with processing any one file. Per file, all modified parity is stored in a temp location until the entire file has been processed, then it is flushed to the real parity at the end. Smaller files can use RAM for this temp storage, but the temp storage for larger files must be placed on disk. </p> http://www.vilett.com/disParity/forum/topic.php?id=57&page=2#post-809 Tue, 03 May 2011 06:20:37 +0000 protovision 809@http://www.vilett.com/disParity/forum/ <p>hey all,</p> <p>I'm having an issue with DSM, it seems to ignore the options I set:</p> <p>- change temp ram (1200)<br /> - change temp folder (e:\temp, 1.5 GB ramdisk)<br /> - show worker</p> <p>when I set these, and then display options again, it has reset my selection. Also, my hidden files are ignored even though the checkbox is clear, and I don't see the effects of 'show worker', if its suppose to show something.</p> <p>anyone else?</p> <p>Also, how is the TEMP RAM and TEMP Folder used? I have them assigned as 1200MB and pointing to a 1.5GB ramdisk. The files I'm protecting are usually 8-20GB each. Is setting the ram high and pointing to a ramdisk helpful?</p> <p>system:<br /> win7 x64, asus p5b-vm, 4x2TB data, 1x2TB parity (esata)</p> <p>thanks! </p> http://www.vilett.com/disParity/forum/topic.php?id=135#post-800 Tue, 15 Mar 2011 17:33:00 +0000 Roland 800@http://www.vilett.com/disParity/forum/ <p>disParity is not aware of whether a disk is basic or dynamic; it does not operate on drives at that level. It operates at the file system level. So I don't see any reason why you would need to convert your dynamic disks back to basic ones.</p> <p>As to your second question, it depends on how your data has changed. If only new files have been added since your last update, then only the newly added file(s) at are at risk of loss should a drive fail; all older data is safe. However, if existing files have been edited or deleted since the last update, then you are definitely in danger of not being able to perform a full restore should a drive fail. It depends on how much of the failed drive overlaps with the missing or changed files in the parity set. </p> http://www.vilett.com/disParity/forum/topic.php?id=135#post-799 Tue, 15 Mar 2011 13:07:11 +0000 Duff26 799@http://www.vilett.com/disParity/forum/ <p>I just got rid of my software raid but now some disks are basic and some are dynamic. I cannot convert back to basic unless I format the drives which I would prefer not to. Does disParity have any problem with the differences in drive configurations like that? Also I understand that there is a parity disk and the parity is stored there, but what if multiple drives change data and disParity has not scanned and one of the disks fails. Will the disk be brought back to the previous state when scanned or be corrupt because the parity calculated does not match the new data on some of the other drives?</p> <p>Thanks! </p> http://www.vilett.com/disParity/forum/topic.php?id=134#post-798 Thu, 24 Feb 2011 17:09:14 +0000 Roland 798@http://www.vilett.com/disParity/forum/ <p>No, deletes will not change the size of previously generated parity, but they do leave "holes" in the parity files that will be filled by later adds on that same drive, if they can fit.</p> <p>To reduce the total parity size after a delete, disParity would have to perform the equivalent of defragmentation pass on the entire parity set, which would be prohibitively expensive.</p> <p>If you *really* want the parity size to reduce after deleting a lot of files, you can always run the "create" command again and build a new parity set. </p> http://www.vilett.com/disParity/forum/topic.php?id=134#post-797 Thu, 24 Feb 2011 13:47:48 +0000 airjrdn 797@http://www.vilett.com/disParity/forum/ <p>I've been testing disParity lately and noticed that when I deleted ~25G of data from my largest dataX, the total size of the parity store didn't decrease. Should it have?</p> <p>For example, I have the following config:</p> <p>parity=G:\</p> <p>data1=H:\<br /> data2=I:\<br /> data3=J:\<br /> data4=K:\<br /> data5=L:\<br /> data6=P:\<br /> data7=U:\<br /> data8=D:\Images\</p> <p>G, H, and I are 1TB drives, the rest are smaller. H had the most data, with only a couple of GBs free. G probably had 30G or so free.</p> <p>I ran a create, and a day and a half or so later, it was done (G is an external USB drive, so that was probably the bottleneck). The amount of free space on G was pretty close to the same as H.</p> <p>After that, I deleted 25G or so from H and ran an update. I assumed the free space on G would increase, but it didn't. Should it have? </p> http://www.vilett.com/disParity/forum/topic.php?id=110#post-796 Thu, 24 Feb 2011 10:58:43 +0000 Roland 796@http://www.vilett.com/disParity/forum/ <p>Glad to hear it worked out! </p> http://www.vilett.com/disParity/forum/topic.php?id=110#post-795 Wed, 23 Feb 2011 14:50:25 +0000 paxtrix 795@http://www.vilett.com/disParity/forum/ <p>Well Roland you were correct! I balanced my WHS drives with a little utility and processed a new parity drive without a hitch. Instead of taking approximately 10 days to complete as it did my 1st time by running the update command to add each drive individually, it took just a minute shy of 24 hrs. Needless to say I was ecstatic that all 14 2tb drives process simultaneously as requested without crashing. I will continue to process two parity drives for redundancy or until I need the extra 2tb for the media server. I’ll state it once again, you’ve created a straightforward program that’s just simple to use and understand! For that my friend, I am grateful… Thanks again!! </p> http://www.vilett.com/disParity/forum/topic.php?id=133#post-794 Wed, 23 Feb 2011 14:15:12 +0000 Roland 794@http://www.vilett.com/disParity/forum/ <p>No, there is no maximum number of drives. That's a common point of confusion. I've explained it here on the forum before although I can't find the post at the moment.</p> <p>The key point is this: disParity recovers a lost drive by using the parity drive combined with *all the other drives used in the backup*.</p> <p>This works because parity can be calculated on any number of bits. You add them all up, and you either get an even or an odd number at the end, and represent that with a 0 or a 1. Later on, if one of the bits goes missing (i.e. a drive crashes) you can figure it out again using all the other bits and the parity (even/odd) bit. This works no matter how many original bits there are. </p> http://www.vilett.com/disParity/forum/topic.php?id=133#post-793 Wed, 23 Feb 2011 13:37:09 +0000 Duff26 793@http://www.vilett.com/disParity/forum/ <p>Yes, thank you. I worded the question poorly. I will probably have four 2 TB drives together so I can use one for the parity. There seems like there should be a maximum of how many drives though. If I connect 10 2TB drives how will one 2TB be able to generate 2TB worth of data?</p> <p>Thanks </p> http://www.vilett.com/disParity/forum/topic.php?id=133#post-792 Wed, 23 Feb 2011 12:53:09 +0000 Roland 792@http://www.vilett.com/disParity/forum/ <p>When you say efficiency, are you referring to how much extra disk space is used?</p> <p>To work the best, disParity requires exclusive use of one extra drive that is as large as the largest drive in your array. For example, say you have 5 drives to protect: two 1TB drives, two 1.5TB drives, and one 2TB drive. You'll need one extra 2TB drive in order to protect all of your data from a single drive failure using disParity.</p> <p>It doesn't matter how many data drives you have, only one extra drive is ever needed.</p> <p>disParity makes no modifications to your data drives and does not alter the way your data files are stored in any way.</p> <p>Hope that clears things up! </p>