Image recovery: Recovering fragmented image files from flash memorycards

Discussion in 'Digital Photography' started by Tonny Iversen, Sep 6, 2007.

  1. Hello. I'm trying to figure out if it exists any software that might be able to recover fragmented images from flash memory cards / digital cameras if the card was quick-formatted (i.e. if the card has a zeroed out FAT), or some details about the jpeg algorithm to figure out how difficult development of such would be.

    My main interest is for .jpg/.jpeg files.

    In the situation when the FAT contains only zeroes, recovery of non-fragmented image files seems to be easy for many (or maybe most?/all?) file / image /data recovery software products, but I haven't been able to find any that can recover anything more than the first fragment of the file if the file is fragmented.

    I don't know the specifics of jpg encoding, but maybe someone here knows if the different parts of the image part of the jpg files are numbered in some way, or contains some patterns/fields that can be used to connect later fragments together with the first fragment? If not, maybe something like FFT/Fouier transformations/frequency analysis or something similar/else could be used to do a qualified guess about which fragments belong to each other and maybe present the user with some alternatives?

    I guess that if image files are fragmented on a memory card, the chance is pretty high that the files only has two fragments. At least this is my guess if the card was new or formatted before use, then used to take pictures and erase some in between, and then quick-formatted afterwards. And if the directory entries are intact and most files are defragmented, I guess it could be possible also to do an educated guess about which clusters are fragments belonging to other files. Also, comparing the starting clusters from all files could indicate if the file is actually fragmented, if the difference between the starting cluster of file n and n+1 is less than the number of clusters necessary to store the file. And if the card was new, zero-filled or filled with a specific pattern before use, it should be obvious if a specific cluster was in use or not before the quick-formatting. (Maybe a full formatting fills the cards with a specific pattern? I haven't tried...)

    Does anyone know of any software doing something like this or is anyone considering making something like this? I'm to pressed for time to consider developing something like this right now, but if someone else wants to look into this, I'd be more than happy to be involved in discussions around the development if you want (though not very much since I don't have much time available). And if someone could comment on these issues, it would be great if I or someone else should consider developing something like this.

    Best regards
    Tonny Iversen ()
     
    Tonny Iversen, Sep 6, 2007
    #1
    1. Advertising

  2. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    Tonny Iversen wrote:

    > In the situation when the FAT contains only zeroes, recovery of
    > non-fragmented image files seems to be easy for many (or maybe
    > most?/all?) file / image /data recovery software products, but I
    > haven't been able to find any that can recover anything more than the
    > first fragment of the file if the file is fragmented.


    First you'll have to find out the cluster size, then the directories,
    which give you the starting cluster, name and size of all the files.

    With fragmented files you'll have to combine every starting cluster with
    every other (not yet assigned) cluster on the medium, a very time
    consuming process. Also consider erased files, which can look like valid
    files, up to the point where a cluster was overwritten later. If you
    miss to detect the file as deleted, you'll spend much time in trying to
    find the no longer existing continuation clusters.



    > I don't know the specifics of jpg encoding, but maybe someone here
    > knows if the different parts of the image part of the jpg files are
    > numbered in some way, or contains some patterns/fields that can be
    > used to connect later fragments together with the first fragment? If
    > not, maybe something like FFT/Fouier transformations/frequency
    > analysis or something similar/else could be used to do a qualified
    > guess about which fragments belong to each other and maybe present
    > the user with some alternatives?


    This requires a "map" of all clusters, containing the characteristic
    information of each cluster. I made such a map already myself, for
    restoring segmented text files by manually pairing the ends and begins
    of the clusters. But with image data? At least you'll need an file
    checker, that can determine whether a chain of clusters *might* be a
    valid jpeg file.


    > I guess that if image files are fragmented on a memory card, the
    > chance is pretty high that the files only has two fragments. At least
    > this is my guess if the card was new or formatted before use, then
    > used to take pictures and erase some in between, and then
    > quick-formatted afterwards. And if the directory entries are intact
    > and most files are defragmented, I guess it could be possible also to
    > do an educated guess about which clusters are fragments belonging to
    > other files.


    FAT32 allocates files strictly sequentially, until the medium is full.
    You can assume that no overlapped writes occured, to multiple files at
    the same time. In this case a recovery will work quite well. When files
    have been erased, and the clusters have been reused later, you'll have
    much more problems.

    DoDi
     
    Hans-Peter Diettrich, Sep 7, 2007
    #2
    1. Advertising

  3. Tonny Iversen

    zac.badenoff Guest

    Re: Image recovery: Recovering fragmented image files from flashmemory cards

    Tonny Iversen wrote:
    > Does anyone know of any software doing something like this
    > or is anyone considering making something like this?
    > I'm to pressed for time to consider developing something
    > like this right now, but if someone else wants to look into this,
    > I'd be more than happy to be involved in discussions around
    > the development if you want (though not very much since I don't
    > have much time available). And if someone could comment on these
    > issues, it would be great if I or someone else should consider
    > developing something like this.
    > Tonny Iversen ()


    Tonny,

    'ere ya go matey!

    It's been done so there's no need to try and develop something yourself.

    http://www.ontrackdatarecovery.com/

    Cheers,

    __
    zb
     
    zac.badenoff, Sep 7, 2007
    #3
  4. Tonny Iversen

    Rob Guest

    Re: Image recovery: Recovering fragmented image files from flashmemory cards

    zac.badenoff wrote:

    > Tonny Iversen wrote:
    >
    >> Does anyone know of any software doing something like this
    >> or is anyone considering making something like this?
    >> I'm to pressed for time to consider developing something
    >> like this right now, but if someone else wants to look into this,
    >> I'd be more than happy to be involved in discussions around
    >> the development if you want (though not very much since I don't
    >> have much time available). And if someone could comment on these
    >> issues, it would be great if I or someone else should consider
    >> developing something like this.
    >> Tonny Iversen ()

    >
    >
    > Tonny,
    >
    > 'ere ya go matey!
    >
    > It's been done so there's no need to try and develop something yourself.
    >
    > http://www.ontrackdatarecovery.com/
    >
    > Cheers,
    >
    > __
    > zb


    Someone looking for ontrack recovery

    found a link http://www.lemonshare.net/Software/1782.html
     
    Rob, Sep 7, 2007
    #4
  5. Tonny Iversen

    cmyk Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards


    > FAT32 allocates files strictly sequentially, until the medium is full.
    > You can assume that no overlapped writes occured, to multiple files at the same time. In this case a recovery will work quite
    > well. When files have been erased, and the clusters have been reused later, you'll have much more problems.
    >
    > DoDi


    Hi Hans,

    Under the FAT system, if you delete a file, as one might do with a poor shot, then it's space is simply marked as being available
    for re-use. If that file happens not to have been the last one written, then there is a good chance that either the next file
    written to the media, or the one after that, will be fragmented. For example (in simple terms, ignoring cluster sizes), if the
    deleted file occupied 1000K and the next two files written each occupied 800K, the second of those files would be fragmented into
    200K and 600K segments. Similarly, if the next file written was 1200K, it would be split into 1000K and 200K segments. Given the
    variability of jpeg file sizes and the tendency of many users to delete files when they review them, fragmentation over 3 or more
    clusters is quite possible.

    Cheers
    --
    cmyk
     
    cmyk, Sep 7, 2007
    #5
  6. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    cmyk wrote:

    > Under the FAT system, if you delete a file, as one might do with a poor
    > shot, then it's space is simply marked as being available for re-use. If
    > that file happens not to have been the last one written, then there is a
    > good chance that either the next file written to the media, or the one
    > after that, will be fragmented.


    FAT32 appends files until the end of the medium is reached. This is an
    optimization to prevent file segmentation. Deleted files are overwritten
    only after the medium is full.

    DoDi
     
    Hans-Peter Diettrich, Sep 7, 2007
    #6
  7. Tonny Iversen

    Phil Carmody Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    Hans-Peter Diettrich <> writes:
    > cmyk wrote:
    >
    > > Under the FAT system, if you delete a file, as one might do with a
    > > poor shot, then it's space is simply marked as being available for
    > > re-use. If that file happens not to have been the last one written,
    > > then there is a good chance that either the next file written to the
    > > media, or the one after that, will be fragmented.

    >
    > FAT32 appends files until the end of the medium is reached. This is an
    > optimization to prevent file segmentation. Deleted files are
    > overwritten only after the medium is full.


    I can think of few techniques _more_ likely to cause fragmentation!
    (Assuming that's what you meant by "segmentation".) Every temporary
    file (just think of your browser cache etc.) will create a hole, for
    example. Every hole is a place where the end of one file and the
    start of another, or even worse the middle of a file, will later be
    placed.

    Phil
    --
    Dear aunt, let's set so double the killer delete select all.
    -- Microsoft voice recognition live demonstration
     
    Phil Carmody, Sep 7, 2007
    #7
  8. Tonny Iversen

    cmyk Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    "Hans-Peter Diettrich" <> wrote in message news:...
    > FAT32 appends files until the end of the medium is reached. This is an optimization to prevent file segmentation. Deleted files
    > are overwritten only after the medium is full.
    >
    > DoDi


    FAT32 does not do this - that is a task for the OS or, perhaps, the media controller. Whether a particular camera or memory card
    controller would do this when writing to the media is entirely up to the card or camera's maker. I know of none that have published
    anything to the effect that their's behaves this way. In fact, for digital media, doing so is pointless from an optimisation
    standpoint.

    --
    cmyk
     
    cmyk, Sep 7, 2007
    #8
  9. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    Phil Carmody wrote:

    > I can think of few techniques _more_ likely to cause fragmentation!
    > (Assuming that's what you meant by "segmentation".)


    Right, I meant fragmentation.

    > Every temporary
    > file (just think of your browser cache etc.) will create a hole, for
    > example. Every hole is a place where the end of one file and the
    > start of another, or even worse the middle of a file, will later be
    > placed.


    I suppose that image files are written sequentially, one after another.
    If not, the problem is very hard (NP?).

    DoDi
     
    Hans-Peter Diettrich, Sep 7, 2007
    #9
  10. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    cmyk wrote:

    >> FAT32 appends files until the end of the medium is reached. This is an
    >> optimization to prevent file segmentation. Deleted files are
    >> overwritten only after the medium is full.
    >>
    >> DoDi

    >
    >
    > FAT32 does not do this - that is a task for the OS or, perhaps, the
    > media controller.


    Read the available documentation about the FAT32 related data structures.

    DoDi
     
    Hans-Peter Diettrich, Sep 7, 2007
    #10
  11. Tonny Iversen

    cmyk Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    I did.

    There is nothing in the FAT32 data structure that dictates the behaviour you claim.

    --
    cmyk
    "Hans-Peter Diettrich" <> wrote in message news:...
    > cmyk wrote:
    >
    >>> FAT32 appends files until the end of the medium is reached. This is an
    >>> optimization to prevent file segmentation. Deleted files are
    >>> overwritten only after the medium is full.
    >>>
    >>> DoDi

    >>
    >>
    >> FAT32 does not do this - that is a task for the OS or, perhaps, the
    >> media controller.

    >
    > Read the available documentation about the FAT32 related data structures.
    >
    > DoDi
     
    cmyk, Sep 8, 2007
    #11
  12. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    cmyk wrote:

    > I did.
    >
    > There is nothing in the FAT32 data structure that dictates the behaviour
    > you claim.


    Look at FSInfo.FSI_Nxt_Free.

    DoDi
     
    Hans-Peter Diettrich, Sep 8, 2007
    #12
  13. Tonny Iversen

    cmyk Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    fsi.FSI_Nxt_Free specifies the lowest free cluster - there is nothing requiring this to occur after the last used cluster. It is
    simply there to speed up disk writes on magnetic/optical media.

    --
    cmyk
     
    cmyk, Sep 8, 2007
    #13
  14. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    cmyk wrote:
    > fsi.FSI_Nxt_Free specifies the lowest free cluster - there is nothing
    > requiring this to occur after the last used cluster. It is simply there
    > to speed up disk writes on magnetic/optical media.


    And consequently files are written contiguously, until the medium
    becomes full.

    DoDi
     
    Hans-Peter Diettrich, Sep 8, 2007
    #14
  15. Tonny Iversen

    cmyk Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    "Hans-Peter Diettrich" <> wrote in message news:...
    > And consequently files are written contiguously, until the medium
    > becomes full.
    >
    > DoDi


    Not necessarily if the lowest free cluster occurs before the last occupied cluster.
    --
    cmyk
     
    cmyk, Sep 8, 2007
    #15
  16. Tonny Iversen

    Phil Carmody Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    Hans-Peter Diettrich <> writes:

    > cmyk wrote:
    > > fsi.FSI_Nxt_Free specifies the lowest free cluster - there is
    > > nothing requiring this to occur after the last used cluster. It is
    > > simply there to speed up disk writes on magnetic/optical media.

    >
    > And consequently files are written contiguously, until the medium
    > becomes full.


    Are you sticking by your previous
    """
    Deleted files are
    overwritten only after the medium is full.
    """
    claim?

    If so, consider this sequence:
    1) Create A
    2) Create B
    3) Delete A
    4) Create C
    After stage 3, what is the lowest free cluster? Is it after B,
    or where A was?


    If not, then can you point out where you retracted the claim?
    And maybe why you held that point of view in the first place.
    And what made you change your mind.

    Phil
    --
    Dear aunt, let's set so double the killer delete select all.
    -- Microsoft voice recognition live demonstration
     
    Phil Carmody, Sep 8, 2007
    #16
  17. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    Phil Carmody wrote:

    > Are you sticking by your previous
    > """
    > Deleted files are
    > overwritten only after the medium is full.
    > """
    > claim?


    Yes, of course :-]

    >
    > If so, consider this sequence:
    > 1) Create A
    > 2) Create B
    > 3) Delete A
    > 4) Create C
    > After stage 3, what is the lowest free cluster? Is it after B,
    > or where A was?


    Why bother with the lowest free cluster?

    Of course it's where A has been, but that doesn't affect the write
    pointer, which stays behind B.

    DoDi
     
    Hans-Peter Diettrich, Sep 8, 2007
    #17
  18. Tonny Iversen

    Phil Carmody Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    Hans-Peter Diettrich <> writes:
    > Phil Carmody wrote:
    >
    > > Are you sticking by your previous """
    > > Deleted files are
    > > overwritten only after the medium is full.
    > > """ claim?

    >
    > Yes, of course :-]
    >
    > > If so, consider this sequence:
    > > 1) Create A
    > > 2) Create B
    > > 3) Delete A
    > > 4) Create C
    > > After stage 3, what is the lowest free cluster? Is it after B, or
    > > where A was?

    >
    > Why bother with the lowest free cluster?
    >
    > Of course it's where A has been, but that doesn't affect the write
    > pointer, which stays behind B.


    Each of the above operations is atomic, so what the **** is a write
    pointer? You're barely making sense, so it's hard to argue against
    you.

    The FAT32 system apparently (see posts by other posters upthread)
    writes each new file at the lowest cluster it can. That's from
    where A has been. This is in agreement with all the prior FAT
    systems that I have been familiar with.

    If you can't work out how and where you're wrong given this
    information, I simply don't have the time and patience to spoon
    feed you any more. Perhaps try on comp.i.dont.have.a.clue ?

    Phil
    --
    Dear aunt, let's set so double the killer delete select all.
    -- Microsoft voice recognition live demonstration
     
    Phil Carmody, Sep 8, 2007
    #18
  19. Re: Image recovery: Recovering fragmented image files from flashmemory cards

    Phil Carmody wrote:

    > Each of the above operations is atomic, so what the **** is a write
    > pointer? You're barely making sense, so it's hard to argue against
    > you.


    You have read about FSInfo.FSI_Nxt_Free?

    Isn't that a write pointer?


    > The FAT32 system apparently (see posts by other posters upthread)
    > writes each new file at the lowest cluster it can.


    So you think that the system scans the FAT, each time a cluster has to
    be allocated to an file or directory?

    > This is in agreement with all the prior FAT
    > systems that I have been familiar with.


    Then familiarize with the new features of FAT32, before posting nonsense ;-]

    DoDi
     
    Hans-Peter Diettrich, Sep 9, 2007
    #19
  20. Tonny Iversen

    cmyk Guest

    Re: Image recovery: Recovering fragmented image files from flash memory cards

    > Then familiarize with the new features of FAT32, before posting nonsense ;-]

    And if you took the time to study how FSI_Nxt_Free is implemented, you wouldn't keep posting stuff like this. FSI_Nxt_Free points to
    the lowest available cluster. Nothing more & nothing less. Nowhere is is said that there can't be any used clusters after this. Get
    it?

    --
    cmyk
     
    cmyk, Sep 9, 2007
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Nawak
    Replies:
    1
    Views:
    1,120
    Terry Baranski
    Aug 7, 2004
  2. Digital Image Recovery
    Replies:
    1
    Views:
    1,279
  3. WeeWilly

    Fragmented Files

    WeeWilly, May 22, 2005, in forum: Computer Support
    Replies:
    10
    Views:
    3,835
    Legion
    Jun 15, 2005
  4. Bob H

    Fragmented text OE

    Bob H, May 27, 2005, in forum: Computer Support
    Replies:
    8
    Views:
    588
    Mike Easter
    May 28, 2005
  5. Digital Image Recovery

    Digital Image Recovery and Memory Stick Recovery; Recovery of Lost Photos

    Digital Image Recovery, Sep 10, 2004, in forum: Digital Photography
    Replies:
    0
    Views:
    1,129
    Digital Image Recovery
    Sep 10, 2004
Loading...

Share This Page