CRC vs MD5

[Grincheux]

For my program I am looking for a fast method to compute a unique identifier.
I explain : For comparing images I uses the MD5 but it is slow.
I thought to CRC32/64 to replace it but I don't think that the result could really be unique. It is not its goal.
What do I have to use. Speed is a priority. Actualy I compute the MD5 on the image bits and I make a MD5 of the whole file.
I could use this library http://www.saphir2.com/sphlib/ but replacing a slow algo by an other slow one is not a good idea.
If ther is no quick hash function I accept cryptographic function, there is no security question. Perhaps a XOR?

I need your advice.

@+

[Bitbeisser]

For my program I am looking for a fast method to compute a unique identifier.
I explain : For comparing images I uses the MD5 but it is slow.
I thought to CRC32/64 to replace it but I don't think that the result could really be unique. It is not its goal.
What do I have to use. Speed is a priority. Actualy I compute the MD5 on the image bits and I make a MD5 of the whole file.
I could use this library http://www.saphir2.com/sphlib/ but replacing a slow algo by an other slow one is not a good idea.
If ther is no quick hash function I accept cryptographic function, there is no security question. Perhaps a XOR?

I need your advice.

@+

Well, it's a "pick your poison" kind of situation. You can either have a fast function or one that produces more likely unique hashes...

Theoretically, a CRC64 function would be twice as fast as MD5 (which produces a twice as large hash, 128bit). Any more advanced crytographic hash functions like SHA1/SHA2 is rather going to be slower than faster.

It might all depend on your exact use case what the best option is...

Ralf

[JohnF]

Do you need a unique identifier or are you just comparing files?

John

[Grincheux]

It is for comparing files, and comparing image once it is loaded in memory.
For the two I used MD5, but it is slow.
I have changed for a CRC32 but I don't know if it is the best thing.

[JohnF]

It is for comparing files, and comparing image once it is loaded in memory.
For the two I used MD5, but it is slow.
I have changed for a CRC32 but I don't know if it is the best thing.

Apart from file size, image height and image width comparisons have you considered a byte by byte comparison, it is quite fast. I made a little compare app that did just that and was surprised by the speed.

Or, using an unsigned long long you could add all the bytes and it would be almost a unique identifier, then make other checks.

John

[Grincheux]

and


These are all kinds of comparisons.
But I think that at the image levels it is missing items, because I only have a sort option for the number of colors.

I have just updated the application, Fresh