The SharePoint item being crawled returned an error when requesting data from the web service. ( Error from SharePoint site: Data is Null. This method or property cannot be called on Null values. )
Almost as useless as the "Unknown error" message. We got talking to Microsoft PSS, they were stumped too, and are currently poring over our log files and trace dumps. In the meantime I think I have found the code that was causing this error.
In one of our features, we use an event receiver to create a number of security groups. The following line was in there:
Web.SiteGroups.Add(groupName, owner, null, null);
Web is an SPWeb, groupName is a string and owner is an SPPrincipal. These are not relevant here. The third parameter in the Add method is the "default user". It was not very clear to me what this was, and since we just wanted empty security groups that the client can fill when we deliver the product, I thought it would be fine to leave that parameter as null. While the code compiles, runs and as far as I can tell creates the security groups just fine, it is what causes that nasty error in the crawl log. I should mention that the error actually causes the crawl to fail and thus breaks search altogether, so it's a big one.
Once I changed the third parameter to a valid SPUser, all was well again.
If you get this error message, I strongly advise you to check what SharePoint API calls you are making that are passing null as a parameter, and then try to eliminate them one by one.
It seems that the "default user" parameter is not the culprit here after all, but the fourth parameter is the one that actually causes this error. This is a description parameter, and even an empty string is OK. See this Microsoft Support article http://support.microsoft.com/kb/2323206