disParity » General Discussion

Added 500GB of files, not all files are actually being added, some even removed?

(32 posts)
1 2

  1. Neostim
    Member

    Glad to see we are not the only ones then.

    I'll have to do more testing, but maybe it is something in common with the newest version of disParity?

    I never had a problem, but I was running .16 for the longest time, then upgraded about a month ago.

    Posted 6 months ago #

  2. rust0r
    Member

    Phatty: that sounds like a similar issue to the one I am having. If you wouldn't mind tracking one of the files and posting it's progress through the two disparity updates so we can see if the same thing is happening (just as I have done below)

    for example, renaming episode 5 of season 8 from tv show 24:

    Detects the rename:

    TV Shows\24\24 - Season 8\24.S08E05.800PM-900PM.HDTV.XviD-FQM.avi moved to TV Shows\24\24 - Season 8\24 - S08E05 - 800PM-900PM.avi

    Removes the renamed file ?

    Removing H:\TV Shows\24\24 - Season 8\24 - S08E05 - 800PM-900PM.avi...

    It is only upon running the update for a second time that an add occurs:

    Adding H:\TV Shows\24\24 - Season 8\24 - S08E05 - 800PM-900PM.avi...

    Posted 6 months ago #

  3. rust0r
    Member

    The common denominator of everyone here, I believe to be disparity 0.20 and possibly Windows 7 64bit (Phatty, what OS are you running?). Neostim stated he never had problems prior to windows 7, but he also at that time upgraded to 0.20; so there are two variables as to why he is having the problem (same as myself).

    Is anyone here that is experiencing the problem, NOT running windows 7?

    Is anyone here that is experiencing the problem, running a version OTHER THAN 0.20 ?

    Lastly,

    Is anywhere here running 0.20 and windows 7, and has ZERO problems ?

    Just trying to see if we can all, between us, isolate what the problem variable might be,

    Thanks!

    Posted 6 months ago #

  4. Phatty2x4
    Member

    Sure:

    1st run

    Scanning D:\...
    NAS\going going gone\Cabin Fever 2.mkv moved to NAS\Movies\Horror\Cabin Fever 2\Cabin Fever 2.mkv
    Adds: 16 Moves: 1 Deletes: 1

    Processing deletes...
    Removing D:\NAS\Movies\Horror\Cabin Fever 2\Cabin Fever 2.mkv from blocks 53208 to 72856...
    Saving file data for D:\...
    5795 records saved in 0.08 sec
    1 file (1.20 GB) removed in 77.09 sec

    2nd run
    Scanning D:\...
    Adds: 1 Moves: 0 Deletes: 0
    Processing adds...
    Adding D:\NAS\Movies\Horror\Cabin Fever 2\Cabin Fever 2.mkv to blocks 19138326 to 19157974...
    Saving file data for D:\...
    5812 records saved in 0.05 sec
    1 file (1.20 GB) added in 55.16 sec

    Posted 6 months ago #

  5. rust0r
    Member

    Phatty, thank you, it looks like we are having the same problem. I may have missed it before, but can you confirm that you are running 0.20 and windows 7 64 ? (if not, what version/os?)

    Posted 6 months ago #

  6. Neostim
    Member

    I just ran another disParity update, and am just confirming, I have the EXACT same issue as you two.

    I'm tempted to run disParity .16 and see if it still does it, I assume .16 would work fine with my .20 created parity. Roland?

    Posted 6 months ago #

  7. Roland
    Key Master

    yes you can "downgrade" to 0.16, it will work with the parity from 0.20. I don't think the disParity file formats have ever changed, not even since the first release.

    I would definitely not be surprised if this bug is new in 0.20, since I changed the upgrade algorithm quite a bit in that release (actually, in 0.17) and we've never heard of this issue before then.

    Posted 6 months ago #

  8. Phatty2x4
    Member

    rust0r - I'm actually using Windows XP SP3 with Disparity 0.20

    Roland - version 0.20 came out roughly on 01/10/2010. That's over month. I know with my set up, I would have never noticed except for (1) the updates were taking an incredible amount of time to run, forcing me to discover a dumb mistake with a junction that linked all the data essentially back on to itself (2) the persistence on this thread which made me look over my log files to try and help them - that's when I discovered the issue with the move not being processed.

    It's completely possible that only a select few have noticed any issues - the reason I would surmise is because of the amount of data that is added to the parity set - i.e. more changes and adds - more log files and disparity to view - leading to people noticing unusual activity in how version 0.20 is operating.

    Posted 6 months ago #

  9. rust0r
    Member

    Thanks for the reply Roland, I was suspect of either 0.20 or win7 64 but based on Phatty's reply stating he is running Win XP that really removes win7 as being the problem. I might downgrade to 0.16 until a future release in that case (if there is one of course).

    As Phatty said, I'm quit confident many are either not running 0.20 or simply haven't paid enough attention to what is happening to notice the problem. I've gone through so many issues with attempted raid5 implementations (freenas, ubuntu, windows) in the process of establishing a stable setup for my home server; that I am now VERY anal about ensuring that everything is on the up and up when it comes to data backup integrity.

    I think this also rules out junctions (if properly established) as being an issue; Neostim is NOT running any sort of junction and has the exact problem that Phatty and myself have.

    Thanks again for your time Roland; and to everyone else that assisted in narrowing down the problem. I'll post up my results, whether I recreate with 0.2 or 0.16; not quite sure what I want to do yet.

    Posted 6 months ago #

  10. freehand
    Member

    I just went back and checked my logs and it looks like I am having this issue aswell. I am using version 0.20 on Server 2008 x64 and when I rename files it removes them on the first update and then doesn't add them back till the 2nd update.

    I didn't really notice this problem because I rarely rename files. Mostly adds.

    I highly doubt it is related to the operating system. .NET is very good taking care of that. Looks like a bug, hopefully roland can fix it when he gets some time. I know work takes priority.

    Posted 6 months ago #

  11. rust0r
    Member

    Just want to update everyone; last night I downgraded to 0.16 and re-created my parity.

    Almost 4TB protected in just under 5 hours, all files have been covered by the initial create. I then renamed files and ran the update; it picked them up (0 adds, 6 moves, 0 deletes). It looks like downgrading to 0.16 fixed the issues we are all experiencing (as Roland states the algorithm for updates changed in 0.17, so any pre-0.17 versions should be acceptable).

    If you are find with running the update twice, I don't see why 0.20 won't work, but it may result in large space discrepancies (at least I was experiencing about a 20GB discrepancy over 1TB data and parity drives)

    Freehand: thanks for the reply, so far we have people experiencing the problems under windows 7, XP and now server 2008. Based on that and Roland's comment about the algorithm for updates it definitely appears to be limited to 0.17 and newer. I would be interested to see if anyone is running 0.17-0.19 and having these issues as well; or if it is JUST 0.20.

    I'm sure any information we can provide will help Roland isolate the issue in future updates and save him a little troubleshooting time.

    Posted 6 months ago #

  12. Roland
    Key Master

    Thanks everyone for all the detailed reports and observations. I now have a theory, at least, about what is happening. I'm hot on the trail... by the way, I now suspect this bug was actually introduced in 0.20, when I added the option to ignore hidden files.

    Posted 6 months ago #

RSS feed for this topic

1 2

Reply

You must log in to post.