It took me a few minutes to find the solution to this so I thought I'd post it on the web for others to find.
If you use the System.Drawing.Image.FromFile method on a file that you do not have sufficient permission to, you will get a "Out of memory" exception. That's right, you would expect a security exception here but that's not the case.
I wish I could remember the forum where I saw this solution so I could give credit. In any case, create a FileStream object from your file and then use the FromStream method on the Image object. This will give you much more intelligent exceptions.