UPDATE: I have been doing some more testing and I am coming to some results that are very different from my initial quick test here. I will have a post about it soon, but for now I'll say that the performance seems to be more than acceptable.
Not that long ago, I was happy to post that I figured out a way to run Linq to SharePoint for an anonymous user. I have some bad news for those who want to use it. I had a quick chat with Waldek Mastykarz the other day at a DIWUG event and I realized why he wasn't quite as excited about my solution as I was. There is quite a performance hit incurred when switching to a secure context, and my solution requires that this is done for all operations, including read operations. With the old CAML query approach, reads can be done without the context switch.
Today I wanted to get an idea of how much this performance hit really would be. So I put together a very quick (and probably not perfect) test. I won't post the code here just yet, but suffice to say that one section of the code ran the Linq query with my "solution" and the other just ran a CAML query the old fashioned way. Each section ran 100 times and used a stopwatch to measure execution time.
First page load - so JIT stuff has to happen here - so perhaps not fair
 0009: 2010-05-26 18:49:27.466 [CBI] 100 Linq Queries took 6000 milliseconds
 0009: 2010-05-26 18:49:27.472 [CBI] 100 CAML Queries took 2 milliseconds
Subsequent page refreshes
 0009: 2010-05-26 18:49:49.303 [CBI] 100 Linq Queries took 4271 milliseconds
 0009: 2010-05-26 18:49:49.307 [CBI] 100 CAML Queries took 3 milliseconds
 0009: 2010-05-26 18:50:03.791 [CBI] 100 Linq Queries took 4973 milliseconds
 0009: 2010-05-26 18:50:03.796 [CBI] 100 CAML Queries took 3 milliseconds
 0013: 2010-05-26 18:51:03.388 [CBI] 100 Linq Queries took 4445 milliseconds
 0013: 2010-05-26 18:51:03.392 [CBI] 100 CAML Queries took 3 milliseconds
From this I would say that there is a <cough>significant</cough> performance issue with the approach I hacked up. I would say that you forget about using it unless someone can come up with some way to cache the datacontext (I tried this and failed so far) or some other clever solution.
The reason I did not post the code here is that it's a quick hack of code that is part of a client web site that I am working on, and I don't want to breach any contract, etc. I also think that a blog post is coming soon about Linq performance in SharePoint in general, where I will do some more rigorous testing.
Sorry for the bad news.