File sync tool for Linux

Discussion in 'NZ Computing' started by Ray Greene, Aug 31, 2006.

  1. Ray Greene

    Ray Greene Guest

    Does anyone know of a file sync tool for Linux?
    I need to clone a Windows fileserver drive to a Linux backup drive, and
    it needs to delete any files on the backup drive that no longer exist on
    the fileserver.

    I tried rsync but it gives an error ("IO error encountered") and won't
    delete anything from the backup drive. A number of people have posted the
    same error on various newsgroups but no one seems to have a solution.

    I could do it with xxcopy from the fileserver, but I would prefer to run
    it from the Linux machine.

    --
    Ray Greene
    Ray Greene, Aug 31, 2006
    #1
    1. Advertising

  2. Ray Greene

    Fred Dagg Guest

    On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene <>
    exclaimed:

    >Does anyone know of a file sync tool for Linux?
    >I need to clone a Windows fileserver drive to a Linux backup drive, and
    >it needs to delete any files on the backup drive that no longer exist on
    >the fileserver.
    >
    >I tried rsync but it gives an error ("IO error encountered") and won't
    >delete anything from the backup drive. A number of people have posted the
    >same error on various newsgroups but no one seems to have a solution.
    >
    >I could do it with xxcopy from the fileserver, but I would prefer to run
    >it from the Linux machine.


    I know it's not what you are asking for, but it sounds like SyncToy
    running on the Windows box, backing up to the backup drive over the
    LAN, would be your best solution.

    Nice and easy to use, and works really well (and is free, if that has
    a bearing).

    http://www.microsoft.com/windowsxp/using/digitalphotography/prophoto/synctoy.mspx
    Fred Dagg, Aug 31, 2006
    #2
    1. Advertising

  3. Ray Greene

    Earl Grey Guest

    Ray Greene wrote:
    > Does anyone know of a file sync tool for Linux?
    > I need to clone a Windows fileserver drive to a Linux backup drive, and
    > it needs to delete any files on the backup drive that no longer exist on
    > the fileserver.
    >
    > I tried rsync but it gives an error ("IO error encountered") and won't
    > delete anything from the backup drive. A number of people have posted the
    > same error on various newsgroups but no one seems to have a solution.
    >
    > I could do it with xxcopy from the fileserver, but I would prefer to run
    > it from the Linux machine.
    >

    http://www.cis.upenn.edu/~bcpierce/unison/
    Its in various distros
    http://packages.debian.org/stable/net/unison
    Earl Grey, Aug 31, 2006
    #3
  4. Ray Greene

    Ray Greene Guest

    On Thu, 31 Aug 2006 18:05:00 +1200, Earl Grey <> wrote:

    >Ray Greene wrote:
    >> Does anyone know of a file sync tool for Linux?
    >> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >> it needs to delete any files on the backup drive that no longer exist on
    >> the fileserver.
    >>
    >> I tried rsync but it gives an error ("IO error encountered") and won't
    >> delete anything from the backup drive. A number of people have posted the
    >> same error on various newsgroups but no one seems to have a solution.
    >>
    >> I could do it with xxcopy from the fileserver, but I would prefer to run
    >> it from the Linux machine.
    >>

    >http://www.cis.upenn.edu/~bcpierce/unison/
    >Its in various distros
    >http://packages.debian.org/stable/net/unison


    I looked at Unison but it appears to sync both ways. I need it to only work
    in one direction. Do you know if it can do that?

    --
    Ray Greene
    Ray Greene, Aug 31, 2006
    #4
  5. Ray Greene

    Ray Greene Guest

    On Thu, 31 Aug 2006 17:36:14 +1200, Fred Dagg <> wrote:

    >I know it's not what you are asking for, but it sounds like SyncToy
    >running on the Windows box, backing up to the backup drive over the
    >LAN, would be your best solution.
    >
    >Nice and easy to use, and works really well (and is free, if that has
    >a bearing).
    >
    >http://www.microsoft.com/windowsxp/using/digitalphotography/prophoto/synctoy.mspx


    Thanks for the suggestion Fred. If I have to run it from the Windows machine
    I'll use xxcopy as I'm already familiar with it. Besides, real geeks do
    things the hard way whenever possible... :)

    --
    Ray Greene
    Ray Greene, Aug 31, 2006
    #5
  6. In message <ed5s8u$sns$>, Ray Greene wrote:

    > Does anyone know of a file sync tool for Linux?
    > I need to clone a Windows fileserver drive to a Linux backup drive, and
    > it needs to delete any files on the backup drive that no longer exist on
    > the fileserver.
    >
    > I tried rsync but it gives an error ("IO error encountered") and won't
    > delete anything from the backup drive.


    At what point does it give that error? Is it at the initial "building file
    list" stage, or while actually transferring some file?
    Lawrence D'Oliveiro, Aug 31, 2006
    #6
  7. Ray Greene

    Ray Greene Guest

    On Thu, 31 Aug 2006 19:50:31 +1200, Lawrence D'Oliveiro
    <_zealand> wrote:

    >In message <ed5s8u$sns$>, Ray Greene wrote:
    >
    >> Does anyone know of a file sync tool for Linux?
    >> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >> it needs to delete any files on the backup drive that no longer exist on
    >> the fileserver.
    >>
    >> I tried rsync but it gives an error ("IO error encountered") and won't
    >> delete anything from the backup drive.

    >
    >At what point does it give that error? Is it at the initial "building file
    >list" stage, or while actually transferring some file?


    It's at the beginning.

    --
    Ray Greene
    Ray Greene, Aug 31, 2006
    #7
  8. Ray Greene

    Enkidu Guest

    Ray Greene wrote:
    > Does anyone know of a file sync tool for Linux? I need to clone a
    > Windows fileserver drive to a Linux backup drive, and it needs to
    > delete any files on the backup drive that no longer exist on the
    > fileserver.
    >
    > I tried rsync but it gives an error ("IO error encountered") and
    > won't delete anything from the backup drive. A number of people have
    > posted the same error on various newsgroups but no one seems to have
    > a solution.
    >
    > I could do it with xxcopy from the fileserver, but I would prefer to
    > run it from the Linux machine.
    >

    I'm using robocopy from the Windows box to do a similar task. How are
    you accessing the Windows drives from the Linux box?

    Cheers,

    Cliff
    Enkidu, Aug 31, 2006
    #8
  9. In message <ed650n$sns$>, Ray Greene wrote:

    > On Thu, 31 Aug 2006 19:50:31 +1200, Lawrence D'Oliveiro
    > <_zealand> wrote:
    >
    >>In message <ed5s8u$sns$>, Ray Greene wrote:
    >>
    >>> I tried rsync but it gives an error ("IO error encountered") and won't
    >>> delete anything from the backup drive.

    >>
    >>At what point does it give that error? Is it at the initial "building file
    >>list" stage, or while actually transferring some file?

    >
    > It's at the beginning.


    Have you tried it with -vvv (at least 3 v's) to enable lots of debugging
    messages? That might give a clue as to the reason for the error.
    Lawrence D'Oliveiro, Sep 1, 2006
    #9
  10. Ray Greene

    Gordon Guest

    On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene wrote:

    > Does anyone know of a file sync tool for Linux?
    > I need to clone a Windows fileserver drive to a Linux backup drive, and
    > it needs to delete any files on the backup drive that no longer exist on
    > the fileserver.
    >
    > I tried rsync but it gives an error ("IO error encountered") and won't
    > delete anything from the backup drive. A number of people have posted the
    > same error on various newsgroups but no one seems to have a solution.
    >

    From memory rsync, the programme needs to be on *both* machines.
    Gordon, Sep 1, 2006
    #10
  11. In message <>, Gordon wrote:

    > On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene wrote:
    >
    >> Does anyone know of a file sync tool for Linux?
    >> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >> it needs to delete any files on the backup drive that no longer exist on
    >> the fileserver.
    >>
    >> I tried rsync but it gives an error ("IO error encountered") and won't
    >> delete anything from the backup drive. A number of people have posted the
    >> same error on various newsgroups but no one seems to have a solution.
    >>

    > From memory rsync, the programme needs to be on *both* machines.


    I was assuming that Gordon wasn't that stupid, that he had the Windows drive
    mounted on the Linux box as a Samba client, so he was using rsync locally
    between two volumes on the Linux box.
    Lawrence D'Oliveiro, Sep 1, 2006
    #11
  12. Ray Greene

    Ray Greene Guest

    On Fri, 01 Sep 2006 16:27:57 +1200, Lawrence D'Oliveiro
    <_zealand> wrote:

    >In message <ed650n$sns$>, Ray Greene wrote:
    >
    >> On Thu, 31 Aug 2006 19:50:31 +1200, Lawrence D'Oliveiro
    >> <_zealand> wrote:
    >>
    >>>In message <ed5s8u$sns$>, Ray Greene wrote:
    >>>
    >>>> I tried rsync but it gives an error ("IO error encountered") and won't
    >>>> delete anything from the backup drive.
    >>>
    >>>At what point does it give that error? Is it at the initial "building file
    >>>list" stage, or while actually transferring some file?

    >>
    >> It's at the beginning.

    >
    >Have you tried it with -vvv (at least 3 v's) to enable lots of debugging
    >messages? That might give a clue as to the reason for the error.


    No I haven't. I'll give that a try tomorrow.

    --
    Ray Greene
    Ray Greene, Sep 1, 2006
    #12
  13. Ray Greene

    Ray Greene Guest

    On Fri, 01 Sep 2006 09:36:05 +1200, Enkidu <> wrote:

    >Ray Greene wrote:
    >> Does anyone know of a file sync tool for Linux? I need to clone a
    >> Windows fileserver drive to a Linux backup drive, and it needs to
    >> delete any files on the backup drive that no longer exist on the
    >> fileserver.
    >>
    >> I tried rsync but it gives an error ("IO error encountered") and
    >> won't delete anything from the backup drive. A number of people have
    >> posted the same error on various newsgroups but no one seems to have
    >> a solution.
    >>
    >> I could do it with xxcopy from the fileserver, but I would prefer to
    >> run it from the Linux machine.
    >>

    >I'm using robocopy from the Windows box to do a similar task. How are
    >you accessing the Windows drives from the Linux box?


    I'm using Samba, I have the fileserver drive mounted on the Linux machine.

    --
    Ray Greene
    Ray Greene, Sep 1, 2006
    #13
  14. Ray Greene

    Ray Greene Guest

    On Fri, 01 Sep 2006 17:43:37 +1200, Lawrence D'Oliveiro
    <_zealand> wrote:

    >In message <>, Gordon wrote:
    >
    >> On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene wrote:
    >>
    >>> Does anyone know of a file sync tool for Linux?
    >>> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >>> it needs to delete any files on the backup drive that no longer exist on
    >>> the fileserver.
    >>>
    >>> I tried rsync but it gives an error ("IO error encountered") and won't
    >>> delete anything from the backup drive. A number of people have posted the
    >>> same error on various newsgroups but no one seems to have a solution.
    >>>

    >> From memory rsync, the programme needs to be on *both* machines.

    >
    >I was assuming that Gordon wasn't that stupid, that he had the Windows drive
    >mounted on the Linux box as a Samba client, so he was using rsync locally
    >between two volumes on the Linux box.


    That's me, not Gordon. The degree of my stupidity is often debated, but I do
    have it mounted as you said :)

    I found a tool called Synchronex which might do the job but it looks a bit
    ugly and it won't sync 2 drives if they are already out of sync to begin
    with. I'm going to try writing my own over the weekend if rsync doesn't tell
    me why it isn't working.

    --
    Ray Greene
    Ray Greene, Sep 1, 2006
    #14
  15. In message <ed8loe$sns$>, Ray Greene wrote:

    > On Fri, 01 Sep 2006 17:43:37 +1200, Lawrence D'Oliveiro
    > <_zealand> wrote:
    >
    >>In message <>, Gordon wrote:
    >>
    >>> On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene wrote:
    >>>
    >>>> Does anyone know of a file sync tool for Linux?
    >>>> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >>>> it needs to delete any files on the backup drive that no longer exist
    >>>> on the fileserver.
    >>>>
    >>>> I tried rsync but it gives an error ("IO error encountered") and won't
    >>>> delete anything from the backup drive. A number of people have posted
    >>>> the same error on various newsgroups but no one seems to have a
    >>>> solution.
    >>>>
    >>> From memory rsync, the programme needs to be on *both* machines.

    >>
    >>I was assuming that Gordon wasn't that stupid, that he had the Windows
    >>drive mounted on the Linux box as a Samba client, so he was using rsync
    >>locally between two volumes on the Linux box.

    >
    > That's me, not Gordon.


    Sorry. Looked at the wrong level of quoting. :)

    > I found a tool called Synchronex which might do the job but it looks a bit
    > ugly and it won't sync 2 drives if they are already out of sync to begin
    > with. I'm going to try writing my own over the weekend if rsync doesn't
    > tell me why it isn't working.


    If you just want to copy newer files over older ones, something like cp -au
    might do the trick.
    Lawrence D'Oliveiro, Sep 1, 2006
    #15
  16. Ray Greene

    Ray Greene Guest

    On Fri, 01 Sep 2006 19:19:28 +1200, Lawrence D'Oliveiro
    <_zealand> wrote:

    >In message <ed8loe$sns$>, Ray Greene wrote:
    >
    >> On Fri, 01 Sep 2006 17:43:37 +1200, Lawrence D'Oliveiro
    >> <_zealand> wrote:
    >>
    >>>In message <>, Gordon wrote:
    >>>
    >>>> On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene wrote:
    >>>>
    >>>>> Does anyone know of a file sync tool for Linux?
    >>>>> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >>>>> it needs to delete any files on the backup drive that no longer exist
    >>>>> on the fileserver.
    >>>>>
    >>>>> I tried rsync but it gives an error ("IO error encountered") and won't
    >>>>> delete anything from the backup drive. A number of people have posted
    >>>>> the same error on various newsgroups but no one seems to have a
    >>>>> solution.
    >>>>>
    >>>> From memory rsync, the programme needs to be on *both* machines.
    >>>
    >>>I was assuming that Gordon wasn't that stupid, that he had the Windows
    >>>drive mounted on the Linux box as a Samba client, so he was using rsync
    >>>locally between two volumes on the Linux box.

    >>
    >> That's me, not Gordon.

    >
    >Sorry. Looked at the wrong level of quoting. :)
    >
    >> I found a tool called Synchronex which might do the job but it looks a bit
    >> ugly and it won't sync 2 drives if they are already out of sync to begin
    >> with. I'm going to try writing my own over the weekend if rsync doesn't
    >> tell me why it isn't working.

    >
    >If you just want to copy newer files over older ones, something like cp -au
    >might do the trick.


    No, we've got around 700GB of files. I copied everything onto the Linux
    machine then someone moved a heap of stuff around on the fileserver so on the
    next backup there were duplicates and the shiny new RAID array is full :-(

    So I need an app that will delete anything on the Linux machine that isn't in
    the same location on the fileserver.

    --
    Ray Greene
    Ray Greene, Sep 1, 2006
    #16
  17. Ray Greene

    Don Hills Guest

    In article <ed8s9l$sns$>, Ray Greene <> wrote:
    >
    >So I need an app that will delete anything on the Linux machine that isn't in
    >the same location on the fileserver.


    Sounds like a job you could do with a few lines of shell scripting language
    using Bash or one of the other CLI shells on the Linux box. I'm not familiar
    with them, but I'll bet someone here is.

    --
    Don Hills (dmhills at attglobaldotnet) Wellington, New Zealand
    "New interface closely resembles Presentation Manager,
    preparing you for the wonders of OS/2!"
    -- Advertisement on the box for Microsoft Windows 2.11 for 286
    Don Hills, Sep 1, 2006
    #17
  18. Ray Greene

    Enkidu Guest

    Ray Greene wrote:
    > On Fri, 01 Sep 2006 19:19:28 +1200, Lawrence D'Oliveiro
    > <_zealand> wrote:
    >
    >> In message <ed8loe$sns$>, Ray Greene wrote:
    >>
    >>> On Fri, 01 Sep 2006 17:43:37 +1200, Lawrence D'Oliveiro
    >>> <_zealand> wrote:
    >>>
    >>>> In message <>, Gordon wrote:
    >>>>
    >>>>> On Thu, 31 Aug 2006 17:33:08 +1200, Ray Greene wrote:
    >>>>>
    >>>>>> Does anyone know of a file sync tool for Linux?
    >>>>>> I need to clone a Windows fileserver drive to a Linux backup drive, and
    >>>>>> it needs to delete any files on the backup drive that no longer exist
    >>>>>> on the fileserver.
    >>>>>>
    >>>>>> I tried rsync but it gives an error ("IO error encountered") and won't
    >>>>>> delete anything from the backup drive. A number of people have posted
    >>>>>> the same error on various newsgroups but no one seems to have a
    >>>>>> solution.
    >>>>>>
    >>>>> From memory rsync, the programme needs to be on *both* machines.
    >>>> I was assuming that Gordon wasn't that stupid, that he had the Windows
    >>>> drive mounted on the Linux box as a Samba client, so he was using rsync
    >>>> locally between two volumes on the Linux box.
    >>> That's me, not Gordon.

    >> Sorry. Looked at the wrong level of quoting. :)
    >>
    >>> I found a tool called Synchronex which might do the job but it looks a bit
    >>> ugly and it won't sync 2 drives if they are already out of sync to begin
    >>> with. I'm going to try writing my own over the weekend if rsync doesn't
    >>> tell me why it isn't working.

    >> If you just want to copy newer files over older ones, something like cp -au
    >> might do the trick.

    >
    > No, we've got around 700GB of files. I copied everything onto the Linux
    > machine then someone moved a heap of stuff around on the fileserver so on the
    > next backup there were duplicates and the shiny new RAID array is full :-(
    >
    > So I need an app that will delete anything on the Linux machine that isn't in
    > the same location on the fileserver.
    >

    robocopy from the fileserver.

    Cheers,

    Cliff
    Enkidu, Sep 1, 2006
    #18
  19. Ray Greene

    Enkidu Guest

    Ray Greene wrote:
    > On Fri, 01 Sep 2006 09:36:05 +1200, Enkidu <> wrote:
    >
    >> Ray Greene wrote:
    >>> Does anyone know of a file sync tool for Linux? I need to clone a
    >>> Windows fileserver drive to a Linux backup drive, and it needs to
    >>> delete any files on the backup drive that no longer exist on the
    >>> fileserver.
    >>>
    >>> I tried rsync but it gives an error ("IO error encountered") and
    >>> won't delete anything from the backup drive. A number of people have
    >>> posted the same error on various newsgroups but no one seems to have
    >>> a solution.
    >>>
    >>> I could do it with xxcopy from the fileserver, but I would prefer to
    >>> run it from the Linux machine.
    >>>

    >> I'm using robocopy from the Windows box to do a similar task. How are
    >> you accessing the Windows drives from the Linux box?

    >
    > I'm using Samba, I have the fileserver drive mounted on the Linux machine.
    >

    smbfs then. I'd still use robocopy or xcopy from the fileserver.

    Cheers,

    Cliff
    Enkidu, Sep 1, 2006
    #19
  20. Ray Greene

    Ray Greene Guest

    On Fri, 01 Sep 2006 22:25:56 +1200, Enkidu <> wrote:

    >robocopy from the fileserver.


    Nope. I refuse to admit defeat on such a simple thing!

    There must be a dozen Windows apps which will do this easily, there's got to
    be a way to do it from Linux. Well there will be soon anyway, this'll be my
    project for the weekend.

    --
    Ray Greene
    Ray Greene, Sep 1, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Trevor Smithson
    Replies:
    5
    Views:
    998
    Blinky the Shark
    May 25, 2005
  2. Trevor Smithson
    Replies:
    1
    Views:
    574
  3. Kue2

    msn search tool & plishing tool

    Kue2, Sep 2, 2005, in forum: Windows 64bit
    Replies:
    1
    Views:
    530
    Andre Da Costa
    Sep 2, 2005
  4. Giuen
    Replies:
    0
    Views:
    747
    Giuen
    Sep 12, 2008
  5. bbiandov
    Replies:
    0
    Views:
    2,010
    bbiandov
    Dec 22, 2008
Loading...

Share This Page