verify your file's MD5 checksums!General


Last Updated:

  1. doogald

    doogald Guides Guide

    An MD5 is what is called a cryptographic hash of a file. Basically, the contents of a file are run through a very specific arithmetic formula, and the result is a particular numeric result of an exact number of bits. What is very useful about the MD5 hash is that it is impossible to predict what the result is going to be before you create the hash, and changing a single bit in the file will create a randomly different MD5 result.

    The MD5 of a file with this sentence: "the quick brown fox jumped over the lazy dog" - is this: bb0fa6eff92c305f166803b6938dd33a

    If I change one bit in that file by capitalizing the first t: "The quick brown fox jumped over the lazy dog" - the MD5 is this: 5c6ffbdd40d9556b73a21e63c3e0e904

    One way that an MD5 is used is to verify that a file that somebody sent to you, or that you downloaded, is the one that you were supposed to have. If the MD5 does not match, you can be certain that somehow the file changed - an incomplete download, or somebody intercepted and changed the file.

    In the particular case of a Nandroid backup, no, there is absolutely nothing that you can do to repair a mismatched MD5 error. As I said before, it is nearly impossible to predict what an MD5 hash result will be before you actually run the calculation, and you cannot take an MD5 hash result and determine the contents of the original file. Just having what the MD5 result should be will not tell you what the original file should have been.

    (And, before any pedants chime in, it is true that MD5 is no longer considered completely secure, as somebody has found a way - using a massively parallel array of PS3s, if I remember correctly - to take a file and add something to it to create a specific MD5 hash result. However, that still is not a solution in this case - the last thing that you would want is to have a Nandroid backup have something random added to it just to match an MD5 hash.)

    The AFV tool will just tell you the result of the MD5 calculation - it will not (nor cannot) tell you what a file needs to be to create a specific MD5 hash result.

    Advertisement
    stephanos and crz6662 like this.
  2. crz6662

    crz6662 Active Member

    Yes, that helps me further, but what could be changing my files ? My backups have never left the phone. I've never even opened them. Let me ask you this. If I mount my SD card to my computer and mess with other files, could that possibly corrupt my backups ? Should I transfer my backups to the computer & just leave them there till I actually needed them ? I'd like to leave them on the phone but not if they're going to keep getting screwed up.

    Let me throw a scenario at you.
    Say I do an MD5 checksum and copy the number shown on a particular backup. Later when I try to reinstall the backup it comes up MD5 mismatch. Technically I could insert the copied number to the mismatched file, replacing the 1 that's there, and it would then install ? Not that I would ever do that. Just trying to wrap my head around this.
    Downloaded your new app version, haven't played with it yet.
    You've been very helpful explaining this for and I think I almost got it. I won't stop till I know it.
    Thanks for the help Scary Alien.
    Doogald: Your explanation also helped. I was in the middle of reply to Scary.
    Thanks Guys
  3. scary alien

    scary alien not really so scary Moderator

    doogald,

    Not sure what you mean by the last part of this...? Can you elaborate a little?

    Thanks!
  4. scary alien

    scary alien not really so scary Moderator

    crz6662,

    Let me back-track a little here first..

    I feel a little like an idiot right now...(lol)...I should have realized and remembered this earlier :eek:.

    I just re-read your first post here in this thread and it occurred to me that there are actually other reasons why you would get an "MD5 mismatch" when trying to do a Nandroid restore.

    The main one is that you have (or had ;)) some non-standard characters in your Nandroid backup directory's name (like a space, ampersand, asterisk, etc.).

    Did you perchance rename your Nandroid backups in ROM manager?

    I have seen this reported by several folks on several threads that this is indeed the behavior that is reported when a Nandroid backup is renamed with "wonky" characters.

    Let me know.
    crz6662 likes this.
  5. doogald

    doogald Guides Guide

    Sure.

    Go into AFV.
    Navigate to where your Nandroids are stored.
    You will see that all of the Nandroid backups are listed as folders - long press on one, try to calculate an MD5, and it will give you a message "Long-press not applicable for selected row".

    (For some reason, I always thought Nandroid backups were zip files, but they really are folders, aren't they?)

    So, never mind, I will go back and re-edit my post and pull that part out.
    crz6662 likes this.
  6. scary alien

    scary alien not really so scary Moderator

    edit: yes, they are folders :)

    lol...no, I don't have any functionality in AFV that operates on folders / directories--just files. Sorry, it is a little confusing...(I'll try to put something in the next version to address this (at least a pop-up note re. that)).

    You'd have to select a system.img, data.img, cache.img, or boot.img file individually in the Nandroid backup's subdirectory or use the Verify Nandroid Backup on the nandroid.md5 file (when you do that, the app will read through the file and checksum pairs in the nandroid.md5 file, re-calculate the MD5 on the file listed, and compare it to the one stored in the nandroid.md5 (it really is what I expect is equivalent to the md5sum -c functionality)).

    Sorry for the confusion re. that. By the way, my next version of AFV will support having a filename passed to it from other apps (like other file managers (already tested with Astro, ES File Explorer, OI File Manager))...thanks to eu1 (again ;)) for that suggestion :).

    Cheers!
    crz6662 likes this.
  7. crz6662

    crz6662 Active Member


    Yes, I did rename all the backups, with the Rom name and date. Had 2 Donut, 2 Eclair & 1 gingerbread. Renamed so I knew which was which.
    I also caught that but had no hash marks but I did have some "spaces".

    Ah ha. A bulb just lite up. Even a single empty space will change my checksum. I think it finally clicked guys.

    Won't be renaming. Period.

    Appreciate you ERIS guys helping this ol' G1 guy out. I'm in the G1 section right now checking it out.
    I think I remember when I 1st joined this forum back in 2008 there wasn't much here so I kinda forgot about it & spent my time over @ XDA.
    Let me say, very helpful people here, even if your in the wrong section. Friendly. Easy going.
    XDA is a great site but a few of them are all over people for things, posting in the wrong area, asking something that was already answered, crap like that. Just a few though. They really need to chill, have a beer, relax.

    Thanks
    scary alien likes this.
  8. scary alien

    scary alien not really so scary Moderator

    Well, the name of the file won't affect the checksum, but the nandroid-mobile.sh script's ability to actually identify your Nandroid backup files will.

    When the custom recovery invokes the nandroid-mobile.sh script, it passes the name of the Nandroid backup directory via a shell command line and a space character will cause the filename to be parsed as separate pieces (i.e., your filename will actually appear short to the nandroid-mobile.sh script and he won't be able to actually do the comparison). His reporting of the error is obviously not as precise as it should be.

    Make sense? Sorry for stringing this out for so long, but at least we got to "meet" a fellow Android user and discuss the merits of MD5 checksums ;).

    Cheers!
    crz6662 likes this.
  9. crz6662

    crz6662 Active Member


    Oh no, not at all. Nice talking to you and I learned some things.
    Think I'll be spending more time here. Makes me want to go out and get an Eris. LOL

    Now let me play with my phone damn it. LOL

    Cheers to you also.
  10. erisuser1

    erisuser1 Well-Known Member

    I looked at the nandroid-mobile.sh script - it would fault out BEFORE any md5sum checks were performed (and give you a different error message) if the "nandroid.md5" file could not be found (for instance, if the backup folder name had some spaces in it).

    I rename nandroid backups all the time - I just avoid the use of anything but letters, digits, dashes, underscores, and the dot (".") character.

    The "nandroid.md5" file only contains the names "boot.img", "system.img", "data.img", et cetera - the only way that you would get a "MD5 mismatch" error if you changed the names of THOSE files (not the folder that contained them).

    It should be easy enough for you to figure out which of those image files is giving you fits - use AFV to examine each of them, and then compare the MD5 checksums one by one to the values reported in the "nandroid.md5" file (it is a simple text file).

    eu1
    crz6662 likes this.
  11. scary alien

    scary alien not really so scary Moderator

    eu1,

    I still think that an errant space in the Nandroid backup directory name will manifest itself as the "MD5 mismatch" condition, but only from having read reports on AF and over at XDA that this ended-up being the case. I thought I had actually tested this myself late last summer, but I don't have that feeling of having first-hand experienced / tested it (possibly since its been so long).

    I've looked at the script before trying to make sense of how this could be and I do certainly hesitate (with very good reason) in disagreeing with you ;) (but I've eaten crow before...:D).

    I will make a point to re-test this when I get home in a little while (just to make sure ;)).

    Also, the AFV's "Verify Nandroid Backup" function invoked via long-pressing the nandroid.md5 file will check each file and report which one does not match, so you don't have to manually do this for each file listed.

    Cheers!

    edit: time for me to eat a little crow: I actually did test this way back in September:

    http://androidforums.com/droid-x-all-things-root/188182-md5-sum-mismatch.html

    and I got an "Error : run 'nandroid-mobile.sh restore' via adb!' error instead. Perhaps the version of ClockworkMod's nandroid-mobile.sh script is different than that of the Eris and will yield this result? Dunno. As you can see, the above thread is from the Droid X forums and involved ClockworkMod and not Amon_RA recovery.

    I'll still re-test this tonight ;).
  12. crz6662

    crz6662 Active Member

    erisuser: Thanks for your input. Helpful explanation. It still baffles me then, what could be screwing up my backups. And this problem just started a few weeks ago, and only, it seems, with my nandroid backups.


    I did download ScaryAlien's newer AFV, which by the way has a nicer look & feel to it. So I'm going to peak around some files, create a new backup and then go inside it and look around.

    Per your explanation it sounds relatively easy so I'm off to search.
    Thanks.
  13. scary alien

    scary alien not really so scary Moderator

    Okay, here's the scoop re. the mysterious "md5sum mismatch" issue.

    I do believe that an errant space (or other character that the shell would find not compatible with his parsing of the command-line arguments) will cause the "Error: md5sum mismatch, aborting" error message to be produced--but probably only for ClockworkMod recovery.

    For Amon_RA recovery, the error message you will see on the phone will be "Error : run 'nandroid-mobile.sh restore' via adb!" (which is exactly what I saw back in September).

    So, what's difference this time? Well, I (finally ;)) checked the /cache/recovery/log file to examine its contents.

    The "Error: md5sum mismatch, aborting" actually occurs in the Amon_RA invocation with an errant space, but only the "Error: run 'nandroid-mobile.sh restore' via adb!" message is actually displayed by Amon_RA's custom recovery. I'm guessing that many others that have reported this behavior (not just Eris users) were using ClockworkMod which can probably display the first message.

    Now, why does this happen? Simple, its because the nandroid-mobile.sh script cannot find the nandroid.md5 file to perform its "md5sum -c" command against.

    Here's what I did to test all of this:

    1. Created a Nandroid backup of my current setup

    2. Renamed the BCDS-20110329-2223 subdir to "gsb 16 -20110329-2223"

    3. I also made a copy of the "gsb 16 -20110329-2223" and named it "gsb16x-20110329-2223" (no spaces) and then I altered the nandroid.md5 file's first entry (for the boot.img file) to place an "x" at the front of his MD5 checksum

    4. Booted back into recovery

    5. Attempted to restore the space-filled "gsb 16 -20110329-2223" and only saw the "Error: run 'nandroid-mobile.sh restore' via adb!" message on the phone's display. But this is what I saw in the /cache/recovery/log for this attempt:

    Code (Text):
    1. Starting recovery on Tue Mar 29 22:34:27 2011
    2. can't open /dev/tty0: No such file or directory
    3. framebuffer: fd 3 (320 x 480)
    4. Build : RA-eris-v1.6.2
    5. [trackball optional]
    6. I:Set boot command "boot-recovery"
    7. Command: "/sbin/recovery"
    8. ro.secure=0
    9. [COLOR=#004dbb][COLOR=#004dbb]:[/COLOR]
    10. [COLOR=#004dbb]: <bulk of the build.prop snipped>[/COLOR]
    11. [COLOR=#004dbb]:[/COLOR]
    12. [/COLOR]adb.connected=
    13. I:Set boot command ""
    14. I:Set boot command ""
    15. I:Set boot command ""
    16. I:Set boot command ""
    17. Restore [COLOR=blue][B]gsb 16 -20110329-2223[/B][/COLOR] ?
    18. Press Trackball or END/PWR to confirm,
    19. any other key to abort.
    20. [COLOR=red]Restoring : .nandroid-mobile v2.2.2: invalid option -- 2[/COLOR]
    21. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 0[/COLOR]
    22. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 1[/COLOR]
    23. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 1[/COLOR]
    24. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 0[/COLOR]
    25. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 3[/COLOR]
    26. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 2[/COLOR]
    27. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 9[/COLOR]
    28. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- -[/COLOR]
    29. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 2[/COLOR]
    30. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 2[/COLOR]
    31. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 2[/COLOR]
    32. [COLOR=red]nandroid-mobile v2.2.2: invalid option -- 3[/COLOR]
    33. nandroid-mobile v2.2.1
    34. Searching for backup directories, matching gsb, to delete or restore
    35. or compress
    36. &#12288;
    37. Looking for the latest backup, will display other choices!
    38.  
    39. Default backup is the latest: /sdcard/nandroid/HT9xxxxxxxxx/gsb 16 -20110329-2223
    40. Other available backups are:
    41. grep: invalid option -- 2
    42. BusyBox v1.15.3 (2010-02-06 17:13:19 CET) multi-call binary
    43. Usage: grep [-HhrilLnqvsoeFEABC] PATTERN [FILE]...
    44. Search for PATTERN in each FILE or standard input
    45. Options:
    46. -H Prefix output lines with filename where match was found
    47. -h Suppress the prefixing filename on output
    48. -r Recurse subdirectories
    49. -i Ignore case distinctions
    50. -l List names of files that match
    51. -L List names of files that do not match
    52. -n Print line number with output lines
    53. -q Quiet. Return 0 if PATTERN is found, 1 otherwise
    54. -v Select non-matching lines
    55. -s Suppress file open/read error messages
    56. -c Only print count of matching lines
    57. -o Show only the part of a line that matches PATTERN
    58. -m MAX Match up to MAX times per file
    59. -F PATTERN is a set of newline-separated strings
    60. -E PATTERN is an extended regular expression
    61. -e PTRN Pattern to match
    62. -f FILE Read pattern from file
    63. -A Print NUM lines of trailing context
    64. -B Print NUM lines of leading context
    65. -C Print NUM lines of output context
    66. &#12288;
    67. Using G1 keyboard, enter a unique name substring to change it and <CR>
    68. or just <CR> to accept: Accepting default.
    69. Restore path: [COLOR=blue]/sdcard/nandroid/HT9xxxxxxxxx/gsb 16 -20110329-2223[/COLOR]
    70. [COLOR=red][B]/sbin/nandroid-mobile.sh: cd: line 997: can't cd to /sdcard/nandroid/HT9xxxxxxxxx/gsb[/B][/COLOR]
    71. [COLOR=red][B]sh: 16: unknown operand[/B][/COLOR]
    72. Verifying backup images...
    73. [COLOR=red][B]md5sum: nandroid.md5: No such file or directory[/B][/COLOR]
    74. [SIZE=3][COLOR=red][B]Error: md5sum mismatch, aborting[/B][/COLOR][/SIZE]
    75. &#12288;
    76. [COLOR=blue][B]Error : run 'nandroid-mobile.sh restore' via adb![/B][/COLOR]
    77.  
    As you can see, the "md5sum mismatch" was produced--just not shown.

    6. Then, I tried to restore my "gsb16x-20110329-2223" with the altered nandroid.md5 file (bad MD5 sum for the boot.img file) and this is what I got:

    Code (Text):
    1. I:Set boot command ""
    2. I:Set boot command ""
    3. I:Set boot command ""
    4. Restore [COLOR=blue][B]gsb16x-20110329-2223[/B][/COLOR] ?
    5. Press Trackball or END/PWR to confirm,
    6. any other key to abort.
    7. Restoring : .
    8. nandroid-mobile v2.2.1
    9. Searching for backup directories, matching [COLOR=blue][B]gsb16x-20110329-2223[/B][/COLOR], to delete or restore
    10. or compress
    11. &#12288;
    12. Looking for the latest backup, will display other choices!
    13.  
    14. Default backup is the latest: [COLOR=blue][B]/sdcard/nandroid/HT9xxxxxxxxx/gsb16x-20110329-2223[/B][/COLOR]
    15. Other available backups are:
    16. &#12288;
    17. Using G1 keyboard, enter a unique name substring to change it and <CR>
    18. or just <CR> to accept: Accepting default.
    19. Restore path: [COLOR=blue][B]/sdcard/nandroid/HT9xxxxxxxxx/gsb16x-20110329-2223[/B][/COLOR]
    20. [COLOR=blue]Verifying backup images...[/COLOR]
    21. [COLOR=red][B].............boot.img: FAILED[/B][/COLOR]
    22. [COLOR=blue]cache.img: OK[/COLOR]
    23. [COLOR=blue]data.img: OK[/COLOR]
    24. [COLOR=blue]system.img: OK[/COLOR]
    25. [COLOR=red]md5sum: WARNING: 1 of 4 computed checksums did NOT match[/COLOR]
    26. [COLOR=red][B]Error: md5sum mismatch, aborting[/B][/COLOR]
    27. [COLOR=red]&#12288;[/COLOR]
    28. [COLOR=red]Error : run 'nandroid-mobile.sh restore' via adb![/COLOR]
    Cheers!
  14. erisuser1

    erisuser1 Well-Known Member

    Wow scary, I'm gonna have to give you a "TL;DR" on that one.

    Actually, it would have been helpful if I did read a little more carefully - crz6662 did say (way back there several novels ago) that he was using ClockworkMod recovery.

    I ran the same experiments as you - put a space in the nandroid backup folder name and attempted a restore, with both Amon_RA and ClockworkMod.

    Just as you said, ClockworkMod produces a (spurious) "MD5 mismatch!" error message, and Amon_RA produces the "run nandroid-mobile.sh" message - on screen


    So, it looks as if crz6662 actually has 5 good Nandroid backups - he just needs to name the backup folders without using any spaces or other unusual characters. Hopefully he didn't go and delete them because (I) convinced him they were bad!


    eu1
  15. scary alien

    scary alien not really so scary Moderator

    lol...I guess it was my turn for a "novel" ;) (haven't had one in a while :)). If I've learned anything from you, its to try to be thorough ;).

    I only saw the "TL;DR" reference in one of your (very) old posts over on XDA a few weeks ago...had no idea what it meant until I Googled it :D.

    Cheers!
  16. crz6662

    crz6662 Active Member


    Yeah, actually I did delete them all. Damn. Oh Well. Not a big deal.
    You guys are great. Really helpful. At least I know what I was doing wrong and how to avoid it next time.
    Thanks again guys
  17. crz6662

    crz6662 Active Member

    Found this zip over @ XDA.
    There's a PDF file that explains how to use this inside the zip, haven't used it yet myself, haven't needed to, but looks relatively easy. Shame I didn't find this earlier.

    Phone backups (Nandroid) have installed with no further problems yet but I've only restored twice, and I haven't changed the backups names at all.

    To help someone else in the future who may need it.

    Later

    Attached Files:

  18. scary alien

    scary alien not really so scary Moderator

    Yeah, this looks like some just detailed the steps for manually fixing / rebuilding a bad or missing nandroid.md5 file.

    However, if the checksums in the nandroid.md5 file truly do not match the entries in the nandroid.md5 file, then, like we said before, you've probably got bigger problems (unwanted space character in the Nandroid backup directory filename notwithstanding) and "fixing" the MD5 sums in the file will not make your files any "better" :p (they didn't match for a reason...and that reason should make you not trust the contents of the .img files that are backed-up in that Nandroid backup directory). At least I wouldn't trust them.

    Use this procedure at your own risk / peril...:eek: ;)
  19. crz6662

    crz6662 Active Member

    Agreed, probably would just delete myself, but an option never the less, for those that would want to attempt.
  20. scary alien

    scary alien not really so scary Moderator

    All,

    I just uploaded my newest version of AFV to the Android Market.

    Been working on this one for a while (sorry it took so long ;)).

    Thanks again to eu1 for suggesting the "launch / send" from other apps idea :).

    Hope you enjoy it!

    Here's the change log:

    version 1.4 - 7-Apr-2011:

    - display elapsed time for long-running operations

    - allow app to be invoked from other applications (tested with Astro File Manager, Estrongs File Explorer, OI File Manager); use the "Send" function

    - added donate button (donate version of app will be published soon)

    - added 'md5sum -c Type Check' function for .md5 files (equivalent to 'Verify Nandroid Backup') to make it clear that functionality exists for non-Nandroid .md5 files too
    crz6662 and doogald like this.
  21. doogald

    doogald Guides Guide

    Well, now this is AWESOME.

    I'll be doing one or the other. Thanks.
    scary alien likes this.
  22. scary alien

    scary alien not really so scary Moderator

    Thank you, doogald! I appreciate both, but your thanks and support is more than enough.

    Keeping two versions of apps is kind of a pain in the keester, so that's why I wanted to update the original one first for everyone.

    Cheers!
    nikalai and kingcast93 like this.
  23. scary alien

    scary alien not really so scary Moderator

    AFV (Android File Verifier) v1.6 just published to the Market.

    Change logs for v1.6 and v1.5:

    [version 1.6 - 02-May-2011]:

    - thanks to all who have donated or purchased the donate version (very much appreciated!)

    - fix FC issue when null/empty URI is received via Send function (please contact me if you experience an FC or other issue)

    - several other minor fixes

    - dynamically identify external storage directory instead of hardcoding /sdcard

    - make pop-up dialogs cancelable via back button


    [version 1.5 - 11-Apr-2011]:

    - fixed FC issue when last remembered directory is no longer present (renamed / deleted) and app is re-launched (thanks to Michael C. for bringing this to my attention)
    doogald likes this.
  24. doogald

    doogald Guides Guide

    Ugh. It took me a month, but I finally did this.
    scary alien likes this.
  25. scary alien

    scary alien not really so scary Moderator

    Thank you very much!

    By the way, the donate version will be updated tonight... I have to rebuild it each time I create a new version and wanted to make sure that there were no issues before rolling things out to the donate users. I like to build and run the donate version to make sure there's no issues (its usually a package name conflict...ugh, I wish there was a better, more dynamic way instead of having to maintain multiple versions).

    I know that sounds kind of backwards, lol, but I thought it would be less impactful.

    Update will be there tonight.

    Cheers and thanks! :)

Share This Page