DNG versus RAW for Archiving

Discussion in 'Digital Photography' started by Mardon, May 13, 2006.

  1. Mardon

    Mardon Guest

    I shoot in Canon RAW with a 20D. I'm archiving my photos in DNG
    using the Adobe DNG converter, without embedding the RAW (CR2) file.
    Does this create any serious disadvantages as opposed to saving the
    original RAW file? I didn't want to take up the space to save both
    and I figure DNG will be more of a 'standard' in the future than CR2.
    Comments?
     
    Mardon, May 13, 2006
    #1
    1. Advertisements

  2. Mardon

    Rudy Benner Guest

    Save both. CDs are cheap.
     
    Rudy Benner, May 13, 2006
    #2
    1. Advertisements

  3. Mardon

    Bill Hilton Guest

    Mardon writes ...
    I think there are disadvantages but others disagree, depending on which
    workflow you use ... first, the early Adobe DNG converters discarded
    some of the EXIF info ... they say this has been corrected for some
    cameras in later versions of the DNG converter, but it's one reason not
    to throw away the original RAW files.

    Second, there are things you can do with the original RAW using other
    converters that you cannot do right now with DNG since several
    converters do not support DNG. If you are only using the Adobe RAW
    converter then it doesn't matter though.

    Two examples of getting useful info from the RAW files that you can't
    get from the DNG files via ACR would include displaying the actual AF
    points used, which can be displayed with the Canon converter, and
    getting the number of shutter actuations, which Capture One displays
    for the Pro cameras. Here's an example of the AF point indicator which
    I can't get from a DNG conversion (the red grids are the actual AF
    points) ... http://members.aol.com/bhilton665/af_africa.jpg ...
    Right now more converters work on .CR2 files than on DNG files. Adobe
    *wants* DNG to become a standard but that doesn't mean it will. You
    won't really *need* DNG until companies quit supporting earlier
    formats. This is already happening with cameras from companies with
    models that did not make the grade, like say the Kodak full-frame
    bodies for Nikon and Canon which won't be supported beyond 2007
    (meaning the existing Kodak converters won't be ported to later
    operating systems ... this is the kind of file that would benefit from
    DNG in a few years when you upgrade your computer OS). But there is no
    sign that mainstream companies like Canon or Nikon care about DNG and
    no sign that they will cease to support earlier file formats when they
    bring out software for later camera models.

    To me, using Canon dSLRs, DNG looks like a solution in search of a
    problem. No way would I delete my RAW files.

    One time I would use DNG is if I got a new model that wasn't yet
    supported by my RAW converter of choice (often it takes weeks or even
    months) ... then once Adobe has support for it I can use the DNG
    converter and open the file in Photoshop, but since I have better RAW
    converters than Photoshop I would consider this a short-term solution
    and definitely wouldn't delete the original RAW files.

    Bill
     
    Bill Hilton, May 13, 2006
    #3
  4. Hello Bill,

    Although this is going a bit off subject, which do you consider is the best
    Raw Converter, if not photoshop?

    Regards,

    Geoff
     
    Geoff. Hayward, May 13, 2006
    #4
  5. Mardon

    tlianza Guest

    Hi,

    If you plan to ONLY use Canon, and you are happy with the available software
    for Canon, then I see no reason to go for DNG. I do believe that DNG will
    outlive these other formats. but there will always be a DNG converter
    available from Adobe.

    I have a number of digital cameras and my workflow is to convert everything
    to DNG. The fact is that photoshop will be around long after Nikon or
    canon's software and it supported by far more consumers. I have no personal
    experience with Canon's software, but three of the professional
    photographers I work with simply won't use it. I use Nikon capture
    occasionally but I find that CS2 is far more efficient.

    I'm sure there are high quality raw converters out there that don't use DNG
    but I think that will change with time. The DNG software development kit is
    free and it is not difficult to use.

    My cameras are Epson RD-1, Olympus E1, Nikon (D2x, d100) and the Leica DMR.
    (To be honest, I haven't converted the D2x files to DNG yet, I'm still
    working with them directly through photoshop) By converting everything to
    DNG first, the workflow is the same for all hardware. The Epson and Olympus
    are kind of odd ball formats, and their raw converters are a bit dated.
    --
    Tom Lianza
    Director of Display and Capture Technologies
    GretagMacbeth LLC
    3 Industrial Drive
    Unit 7&8
    Windham, NH 03087
    603.681.0315 x232 Tel
    603.681.0316 Fax
     
    tlianza, May 13, 2006
    #5
  6. Mardon

    Mardon Guest

    Thanks Bill, and Rudy too. I'm convinced. I think I'll save the
    original RAW files and also a DNG version on separate disks. That
    will cover both bases and also provide protection from a disk
    failure.
     
    Mardon, May 13, 2006
    #6
  7. Benefits of DNG:
    http://www.barry.pearson.name/articles/dng/benefits.htm

    Is DNG safe?
    http://www.barry.pearson.name/articles/dng/safety.htm

    Note: if you only keep one copy of your raws files, (whether native or
    DNG), you are running a risk if your disc fails. If you have 2 copies,
    a common approach is to have one native raw + one DNG.

    Various articles about DNG:
    http://www.barry.pearson.name/articles/dng/
     
    Barry Pearson, May 13, 2006
    #7
  8. Bill Hilton wrote:
    [snip]
    [snip]

    It is interesting that some people see those arguments as the main
    benefits of DNG. I see them as among the least important. I've tried to
    separate the benefits into "tactical" and "strategic" benefits, in the
    following:
    http://www.barry.pearson.name/articles/dng/benefits.htm
     
    Barry Pearson, May 13, 2006
    #8
  9. So far as I can tell, the issue comes down to future software
    support. In 20 years, will it be easier to process DNG or CR2 files?
    Quite possibly both will still be easy. I would think that RAW from a
    20D will be supported for a long time. DNG is newer and it's a new
    *category*, we have little back history to predict from.

    Keep archive copies of your software disks. There will always be
    simulation software that will let you run today's windows and
    photoshop on future hardware and whatever OS it runs, just like you
    can run PDP-8 and DECSYSTEM-20 software in simulators on your PC
    today. If you have archive copies of today's windows, photoshop,
    ACR, and so forth, then you will always be able to open the old files
    even if current versions of Photoshop (or whatever) don't support the
    old formats.

    I'm facing this same question. I've been converting to DNG to save
    storage space (a 2:1 savings with my previous Fuji S2), and with my
    D200 I find that Photoshop CS ACR doesn't support it, so DNG is the
    *only* way I can shoot raw with my new camera. And I wonder about
    deleting the RAW and NEF files. So far I've kept most of them, but
    I'll probably eventually give in and scrag them.
     
    David Dyer-Bennet, May 13, 2006
    #9
  10. Mardon

    Bill Hilton Guest

    Geoff. Hayward writes ...
    I'm a big fan of running your own tests on your own files because there
    are differences in workflow, because some converters work better on
    certain cameras, and because tastes vary on what's the 'best' result.
    I would advise you to test the converter that came with your camera,
    Photoshop or Elements converters (if you have one of these), then
    download the free RawShooter Essentials (RSE) and the free trial
    version of Capture One.

    I tested these looking at four things that are important to me ...
    efficiency in quick-editing large numbers of files on a laptop, how
    easy it was to get the best color, how sharp the detail in the
    fine-edged areas and how smooth the demosaicing in the out of focus
    areas.

    Looking thru a lot of files fast is what I need when traveling with a
    laptop, shooting several GB early, breaking for lunch and an editing
    session, then shooting several GB in the afternoon. Often my wife is
    with me sharing the same laptop and sometimes we are in places with
    limited power so we like to make a quick cull and then write the
    keepers off to two external hard drives. One morning we shot 14 GB
    between us, as the extreme example (more often its 3-5 GB but we had
    lions and cheetahs and birds etc). RSE is the best for this type of
    quick edit because you can set up the slide-show and quickly review
    them. It's better than Capture One for the laptop because the previews
    generate faster and because they are about 1/3 the size (we had over 10
    GB of just preview files in Tanzania not long ago). Photoshop CS and
    the Canon software are very poor at this kind of previewing. For
    different workflows maybe Photoshop is OK but not for this one.

    For colors, Photoshop is OK but I get more pleasing colors with both
    RSE and Capture One. As one example, CO has four different options for
    linear, 'film extra shadow' (somewhat like Astia slide film), 'film
    normal' (somewhat like Provia 100F in saturation) and 'film high
    contrast' (almost like Velvia) ... different options look better on
    different images and you can make those comparisons with a single click
    .... plus you can apply an ICC profile for even more accurate colors ...
    the profiles by Magne Nielson for different lighting conditions are
    excellent for my 1Ds and 1D Mark II, for example. Photoshop offers
    nothing like this. All of these options makes it easier to quickly
    arrive at the best colors compared to Photoshop CS RAW.

    For sharp edges and clean backgrounds ... you'd think these are related
    in that if the converter applies extra sharpening the out of focus
    areas will not be as smooth, but that's not the case. It's a bit
    tricky to compare because some of these apply extra sharpening at
    default settings (RSE comes to mind), but I find that with my workflow
    I get more detail where I need it and less demosaicing artifacts where
    I don't want faux detail with CO than with Photoshop. I test this with
    pics of birds with out of focus backgrounds ... the feathers and eyes
    are sharper in CO yet the oof background is smoother. Test this for
    yourself by blowing them up to 400% and comparing.

    At any rate I rate Capture One as my favorite with RSE close behind
    (considering it's free it's a great bargain). Photoshop is a poor
    third and I almost never use it except for maybe something like
    checking test shots for sensor dust or if I need the vignetting option
    for shots taken at wide apertures with one of the 4:1 zooms, and the
    Canon software is worse than Photoshop. The only reason I keep the
    Canon software loaded is to check EXIF info that the others don't
    recognize, like auto-focus points.

    But you should test for yourself, you may have different criteria or
    things may be different with another camera brand.

    Here's a test similar to mine but looking at different things,
    comparing two Canon programs to jpegs to Photoshop to Capture One for
    the 1D Mark II ...
    http://www.dpreview.com/reviews/canoneos1dmkii/page17.asp

    Bill
     
    Bill Hilton, May 13, 2006
    #10
  11. Mardon

    Bill Hilton Guest

    tlianza writes ...
    I've used the various converters that came with my Canon cameras and I
    agree with this, none were especially good and I don't know many who
    use them. But I think Capture One and RSE and RSP are as far above
    Photoshop CS RAW as CS RAW is above the bundled Canon programs, based
    on tests I've run. I'm not an Adobe basher, I've used Photoshop since
    V 4.0, am an "Adobe Certified Expert" in Photoshop and have read
    Fraser's book "Camera RAW with Adobe Photoshop CS" trying to get the
    best from CS RAW ... heck, I wish it was the best converter, but it's
    not and even the Adobe engineers on the Photoshop NG will concede that.
    I tried out the DNG converter and played with the DNG files and I just
    don't see the attraction ... if I load a .CR2 file in CS RAW and if I
    load a DNG version of the same file in CS RAW they look *exactly* the
    same. So what have I gained? Nothing, except I can't load the DNG
    file in the best converter I own and I can't read the EXIF data I might
    want (if it wasn't deleted during conversion) because the Canon
    software won't load the DNG file. There is no benefit to DNG for me so
    long as Adobe still supports the native formats of the cameras I use
    (except the DNG files are a bit smaller, maybe 10% smaller for my Canon
    files).

    The day that Adobe quits supporting .CR2 files with its RAW converter
    will be the day DNG becomes necessary, if you use CS RAW. How long do
    you think it will be before Adobe quits supporting a popular file
    format like that? 10 years? 20 years? Long time, I'll bet ... once
    they no longer support the native file then DNG is the way to to.

    Bill
     
    Bill Hilton, May 13, 2006
    #11
  12. Mardon

    Alfred Molon Guest

    Just think about it. Are you still able to open files in formats of
    1986? Probably yes, for most formats. Software is usually backwards
    compatible.

    In any case, there is RAW converting software (Dave Coffin's dcraw)
    whose C code you can save and compile in the future, thereby obtaining a
    RAW converter for your camera which runs on a future operating system.
    Most likely C compilers will still be around in 20 years, and even if
    not, somebody in the future might rewrite dcraw's C code into a new
    software language.

    I might add that RAW is a very simple file format - just a dump of pixel
    brightness values. Should not be a problem for a future "archaeologist"
    to recover our 2006 RAW images.
     
    Alfred Molon, May 14, 2006
    #12
  13. It isn't a matter of whether in future you will be able to find SOME
    way of accessing the native raw files of 2006 cameras. They've been
    reverse-engineered and that knowledge will still be around. So will
    Dave Coffin's code, as you point out.

    The question is - will you be able to do so with your future workflow
    and tools of choice, at an acceptable cost? New software products are
    often launched supporting a restricted range of cameras, and may never
    get around to supporting some of the older cameras. You may be using
    such software products in future.

    [snip]
    (Hm! Is TIFF/EP or TIFF 6.0 "a very simple file format"?)

    The delay of support for new cameras typically isn't because of an
    unknown file format, although that is sometimes the case. Much of the
    delay is because a raw converter needs lots more information available
    to it than the raw image data. It needs to know quite a lot about the
    characteristics of the camera & sensor. The risk is that this
    information will not be so easy to find. Software suppliers typically
    obtain one or more instances of the new camera, take some test shots
    under controlled conditions, then identify and build in those
    characteristics to their upgraded product. Will they be able to do this
    in future with today's cameras?

    The key difference with DNG is that those characteristics are built
    into every DNG file. There are about 20 or 30 of them, whose values are
    typically built into the DNG writer, whether that is a camera that
    writes DNG, or a raw converter that saves as DNG, or a DNG Converter,
    whether supplied by Adobe or not. This is intended to make each DNG
    file "self-contained" - not reliant on extra data such as camera
    descriptions & profiles. (It is why ACR 2.4 can handle DNGs for more
    than 50 cameras that were launched after ACR 2.4 was released).

    On the following page, I've illustrated this by showing examples
    extracted from DNG files for about 13 cameras. The parameters are
    documented in the DNG specification, although of course that doesn't
    record the values for specific cameras. (The page exists for a
    different purpose, but it makes this point too).
    http://www.barry.pearson.name/articles/dng/innovation.htm

    Just as important is the fact that DNG can hold extra metadata in an
    openly-documented form (XMP). This can include "Rights management"
    metadata and "Asset management" metadata. (As well as editing and
    settings information of the raw converter, which will probably be less
    useful in future).
     
    Barry Pearson, May 14, 2006
    #13
  14. Hi Bill,

    WOW - that was some answer - many many thanks for the detailed explanation
    which is much appreciated. I have started to use RSE and quite like it
    already, Capture One is pretty obviously a piece of pro software, out of the
    reach of amateurs like me.

    Many thanks again,

    Geoff.
     
    Geoff. Hayward, May 14, 2006
    #14
  15. Mardon

    Alfred Molon Guest

    You might extract that information from the source code of dcraw. Once
    dcraw supports a camera, the specifics of the sensor and the camera are
    documented in dcraw's code.
     
    Alfred Molon, May 14, 2006
    #15
  16. Remember the question is - will you be able to access the images with
    your future workflow and tools of choice, at an acceptable cost? New
    software products may never get around to supporting some of the older
    cameras. You may be using such software products in future.

    Does dcraw hold the specific data which will be needed by those future
    raw converters? I doubt it. (And I also doubt if it holds the specific
    data that DNG itself holds). Will it be possible to exploit the data
    held within the dcraw code and convert it to the form used by those
    future raw converters? And will anyone bother to do so?

    The point about DNG is that the parameters are in a documented format,
    and held in the DNG file itself. Future products that support DNG
    properly will find what they need in the file.
     
    Barry Pearson, May 15, 2006
    #16
  17. Rest assured that it *does*.

    Not only that, it is written in ISO/ANSI Standard C, which
    certainly means that 30 or 50 years from now you *will* be able
    to compile and run a copy of the source code as it is today.
    Yes and yes. But as noted, that is insignificant because you
    will be able to run today's version of dcraw 50 years from now
    on just about any platform, and convert a RAW file from an
    archive saved from today.
    I don't see your point. That does not affect cameras using various
    "raw" formats today.
     
    Floyd L. Davidson, May 15, 2006
    #17
  18. I would like to see evidence of that. dcraw has the data needed by
    dcraw, of course, but that is not necessarily the same thing. I have a
    copy of the dcraw C code, but I find it hard to itemise just what data
    it actually does have, and even if I did I probably wouldn't be able to
    match it against the requirements of those future raw converters.
    But that isn't what I am talking about here. See above: "Remember the
    question is - will you be able to access the images with your future
    workflow and tools of choice, at an acceptable cost?"

    (I said before that "It isn't a matter of whether in future you will be
    able to find SOME way of accessing the native raw files of 2006
    cameras. They've been reverse-engineered and that knowledge will still
    be around. So will Dave Coffin's code, as you point out". This
    sub-thread isn't about using tools that people wouldn't normally
    choose. In a recent survey, only 5% of those surveyed used dcraw at
    least some of the time).
    I would like to see evidence for that, for the same reasons I
    identified above. Frankly, with raw-capable cameras appearing at the
    rate of perhaps 50 a year, I think it is very unlikely that people WILL
    bother to do so in years or decades to come. It is far more likely that
    they will support DNG, and leave people to convert to DNG at some time.


    Look at the current situation. dcraw supports over 200 cameras. ACR
    supports over 100. Others tend to support significantly less. And that
    is for cameras currently in use, or in the recent past. Why doesn't ACR
    support all the cameras that dcraw supports? Perhaps because even for
    Adobe this is too much work, or perhaps dcraw doesn't have the right
    data. Why don't today's other raw converters support the same set even
    as ACR? After all, they could extract the parameters from DNGs if those
    chose to, or from dcraw, or both. But something stops them, or they are
    not motivated to do so. "Critical mass" is important.
    Again, see above! "Will you be able to access the images with your
    future workflow and tools of choice, at an acceptable cost?"
    Yes it does. DNGs archived today will have those parameters stored
    within them for future use.
     
    Barry Pearson, May 15, 2006
    #18
  19. I contains the information required to convert data from the
    various cameras into image formats (PPM and TIFF). What else
    could you want???? (Okay, okay, *ease* of use... :)
    Exactly! Unless you restrict yourself to platforms that are not
    extensible, you'll be able to integrate dcraw into any future
    work flow. (Of course anyone who chooses non-extensible systems
    has far more problems than just that.)
    So you are telling us that you don't read C code well and don't
    understand the concept of camera data being converted to an image?

    I doubt that you can't see the evidence; it appears you are just
    unwilling to admit it.
    Fine. That won't delete every raw data file that exists today,
    and it will not delete every copy of the source code to programs
    that convert those files to image formats.
    Well, there you are! 300 formats won't be lost...
    Choose appropriate tools...
    Cameras today are *not* archiving data in DNGs, with only a hand
    full of exceptions.
     
    Floyd L. Davidson, May 15, 2006
    #19
  20. I asked "Does dcraw hold the specific data which will be needed by
    those future raw converters?" It contains what dcraw needs. Now what
    evidence is there that it contains what those future raw converters
    need?

    This sub-thread is about "will you be able to access the images with
    your future workflow and tools of choice, at an acceptable cost?". Not
    "will you be able to use dcraw in future?" We can assume you will, but
    that isn't the point here.

    [snip]
    "... and tools of choice"! If dcraw isn't a tool of choice, which it
    currently isn't for 95% of raw shooters, then it shouldn't have a part
    to play.

    [snip]
    This sub-thread is about "... future workflow and tools of choice ...".
    And dcraw clearly isn't a typical tool of choice.
    Please don't play such games! See:
    http://www.barry.pearson.name/business/cv.htm

    As I showed, there is currently evidence that the existence of dcraw
    code doesn't result in support by current raw converters. (dcraw
    supports lots more cameras than other current raw converters). So why
    should anyone believe that future raw converters will be more likely to
    exploit dcraw code?

    It is probably just wishful thinking to believe that future raw
    converters will support the native raws of current cameras, given that
    even current raw converters don't always do so. They will do what their
    business plan requires, which will typically be to support the major
    cameras around in future.

    When considering what is needed, some numbers are useful. In January
    2005, Adobe released a version of ACR that supported about 50 cameras.
    In January 2006, the new version of ACR supported about 100 cameras.
    Call it about 50 new cameras with raw capability launched per year.
    They don't all have different file formats, but they typically have
    different profile information in raw converters.

    New raw handling products are steadily appearing, sometimes by adding
    raw support to an existing product, sometimes by introducing a new
    product. As a conservative estimate, I'll assume about 10 such products
    per year. (Given that we appear to have more than 100 such products at
    the moment, I think the rate must have been higher than that up to
    now). Sometimes they build on a component that already has raw support,
    but sometimes they support raw via new code. In the latter case, they
    tend to support the most recent cameras first, and add others later, if
    at all.

    In 10 years time, with perhaps 500 new cameras, and perhaps 100 new raw
    handling products, how likely is it that anyone's tools of choice,
    perhaps including that latest "must have" widget, will support raw
    files for cameras over 10 years old? I don't want to bet on it.

    There is a combinatorial problem - 500 cameras times 100 current
    products plus 100 new products less defunct products. There is the
    business case for any particular product supplier - what it is worth to
    them, compared with their return on investment if they do something
    else instead? (If anyone doesn't like my numbers, replace them with
    your own numbers).
    All you have to do is cite the evidence, instead of trying to distract
    attention from the fact that you don't have it! Just tell us!

    [snip]
    Ah! So the answer to "will you be able to access the images with your
    future workflow and tools of choice, at an acceptable cost?" is "NO -
    unless you let someone else make the choice for you!"

    [snip]
    The DNG Converter IS doing that. And this thread started off discussing
    the DNG Converter: "I'm archiving my photos in DNG using the Adobe DNG
    converter". Whether cameras output DNG is totally irrelevant to this
    thread!
     
    Barry Pearson, May 15, 2006
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.