Same .webpart url in Web Part Gallery causes weirdness

I ran into an interesting issue today, and although I haven't fully tested this yet, I think I now get what is going on.

I was working on refactoring a number of web parts for a customer, they had a less experienced SharePoint dev create a bit of a mess instead of a proper WSP, and so my first task was to create a proper SharePoint project in Visual Studio and include the necessary web parts. No issues, all worked on my clean test environment.

I then proceed to deploy my shiny new WSP on the test server, and while most things work as expected, the web parts that I included are not available. The feature that they are part of is activated so I am stumped for a minute. Quickly I start to suspect that this has something to do with the old web parts, but my new ones are in a different assembly, different namespace etc etc. After some poking around in the xml, I realize that both my web parts and the old versions have the same name, and thus deploy to the same URL in the web part gallery. This can be seen in the elements.xml file for the web part.

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/" >
  <Module Name="MyWebPart" List="113" Url="_catalogs/wp">
    <File Path=MyWebPart\MyWebPart.webpart" Url="MyWebPart.webpart" Type="GhostableInLibrary">
      <Property Name="Group" Value="JoeTest" />
    </File>
  </Module>
</Elements>

A few quick tests on my local machine confirm that such URL collisions will cause problems, on my local machine it looked like web parts were being overwritten by the latest feature to deploy, but on the test server the old web parts persisted. This needs some more testing to fully understand what the intended behavior is, and whether it is something permissions related.

Nevertheless, as soon as I updated my URL deployment location, I was able to deploy the new web parts to the test server and use them on pages.

<?xml version="1.0" encoding="utf-8"?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/" >
  <Module Name="MyWebPart" List="113" Url="_catalogs/wp">
    <File Path=MyWebPart\MyWebPart.webpart" Url="new_MyWebPart.webpart" Type="GhostableInLibrary">
      <Property Name="Group" Value="JoeTest" />
    </File>
  </Module>
</Elements>

I realize that it probably isn't best practice to have web parts with the same name, etc. but trust me that I have a good reason for doing this. In any case, after some thought it makes perfect sense that SharePoint will not allow us to deploy two .webpart files to the same URL, and so a change to the URL is needed to work around this.

If you still don’t ‘get’ what Governance is

True story: About one year ago I went to Microsoft TechEd in Berlin and attended all types of sessions. I was there to learn about technology, and I somewhat accidentally attended a session on SharePoint governance. I didn’t really know what that was, and frankly I couldn’t have cared less. I am a techie, and this was some sort of new fad that the global companies were all raving about. Some sort of corporate mumbo jumbo that was useless when it came down to doing some real work. Not a single line of code in the entire talk, just a bunch of Visio.

Does that sound familiar? Can you identify yourself in that story? If so, I will try to explain to you why this governance thing is being discussed so much, and why you actually do need to be aware of it, even as a hard core techie.

Let’s talk about traffic. I don’t mean network traffic, I mean the kind most of you get into on a daily basis out in the streets. When you get into your car, or I on my bike, there is a lot that goes on between leaving home and getting to work. First, there are all the roads we need to navigate. Then there is the weather that can cause the road conditions to change. Lastly, there are all those other people out there on the road who are also trying to get from point A to point B. It is fair to say that the traffic on our roads today is a complex system, and no one really has a grasp of the ‘overall plan’ for the day. Somehow we all make it to where we need to go most of the time, and when something does go wrong, it usually degrades in a way that is acceptable. By this I mean that fatal accidents are fairly rare as opposed to fender benders, and fender benders are rarer yet when compared to being late due to congestion.

So why am I on about traffic? Because I think this extremely complex system that is used by a huge number of human beings at the same time could not work if it weren’t for governance. There's that word again. Let’s see what I mean by governance in terms of traffic. Those of you who have driver’s licenses should know a good deal of traffic governance, by that I mean your local traffic laws. Every new driver is taught that red lights mean stop, what a yield sign looks like and what it means, and how fast a vehicle may travel on a given road. These rules are what ensures that we all behave in a given way on the road, and thus don’t run into each other very often. Being human means we don’t always get it right, but that’s another story.

There are other rules that the average driver doesn’t know that are just as vital to the success of our daily drive. What type of road surface is used? What is the minimum radius of a turn on the expressway? These are rules the road engineers need to be aware of.

There are also specific situations that warrant their own set of rules and behaviors. When something does go wrong, the emergency response teams have a very well defined protocol to handle that event, ensuring the best possible outcome for those impacted and the quickest possible return to normal conditions.
Lastly, I want to mention the massive differences in how traffic operates in different locations. Compare rush hour in Bangalore http://www.youtube.com/watch?v=0cdUPOvSXOo with rush hour in Berlin http://www.youtube.com/watch?v=6jUc6dHcbCo and rush hour in Utrecht http://www.youtube.com/watch?v=UlQYP4WN-5w. You immediately see there are some localized differences. Some places drive on the right side of the road, some on the left, and some have dedicated lanes for bike traffic. Some places have a visible order and structure to the driver's behavior while others exhibit what looks like chaos to the outside observer. Nevertheless, each is designed with a single purpose to get everyone from point A to point B as safely and quickly as possible.

So what does all this have to do with SharePoint, and specifically SharePoint governance? As mentioned, the traffic system is very complex. What makes it complex however are not the roads or vehicles but all the individual people that interact with it at any given point in time. Each with a different goal in mind, and each not aware of the needs of the others in the system. What makes it work, is the adherence to the rules and guidelines that belong to that system.

If we compare the road system to SharePoint, we can see that there are many similarities. I would argue that the bits provided by Microsoft on the SharePoint DVD are the equivalent of concrete, asphalt, vehicles and traffic lights. They represent a possibility, or opportunity to create a useful (and complex) system. When we start a new SharePoint project, it is vital that we decide on rules and guidelines with which we will use all the components available to us. We may not even want or need all the components at all places. Just because there are thousands of road signs available to a road designer, doesn’t mean it is useful to use each and every one. How helpful is this scenario?

The first step we need to decide is why we are building our SharePoint system to start with. A road crew generally needs to build a road to connect two places, to improve traffic flow in a congested area, or to upgrade a broken down old road. Our SharePoint project needs to be just as clear in purpose. Replacing an old intranet that doesn’t work anymore, connecting two remote offices so they can share information. Those are very clear and simple reasons that make it easy to understand why we are starting a new project. Saying that we need to “improve collaboration in the enterprise” is simply too vague and would be the equivalent of saying that we need to build a road to “enhance the citizens driving experience”. I’d like to see a city allocate budget to that project.

Once we have a purpose, we need to decide on the rules and guidelines that we will follow so that we can achieve our purpose. There will be different types of rules for different scenarios. Some will be very hard and some quite flexible. Road crews can choose which end of the road to start at, or even which sections to work on at what time. They can't however just decide to route the road through a nature reserve because it makes it easier.

The rules will also not be the same for all organizations. We may have a different set of goals and a different culture. In road terms, the Netherlands sets speed limits that are quite absolute. There are many speed cameras and getting caught gong just a few km/h over will result in a fine. In the Toronto area, going 20 km/h over the limit on the expressway will generally not raise any interest from the police. As a matter of fact, I remember it being somewhat rude to go any slower. Culture has a huge impact on the rules we follow and how.

In SharePoint projects, we will have to determine the rules that work for our organization. Some hard rules will need to be set, such as access rights for instance. More flexible rules may include good practices with respect to metadata use and version control. We also need to determine the rules and guidelines for different target groups and different scenarios. Our end users will not be interested in the rules regarding database maintenance, but our IT department will need to be very aware of these. We also need to consider special cases such as system failures, and the procedures that need to be followed in order to respond to these. In the traffic scenario we can think of the highly organized Emergency Response Teams. In SharePoint we can think of disaster recovery plans. How often do you think the EMTs practice their routine, and how often does your project practice their disaster recovery?

Having determined these rules, we should have a good place to plan our project in terms of specifics. How will we build what is needed and who will be involved. This activity happens on most projects today, but it’s critical to know that this plan should be a derivative of understanding why we are building something, and what the desired behavior with that something is. In road terms, we can plan the best roundabout out there, but we need to know that it’s purpose is to slow down driver’s top speed but at the same time not forcing them to stop completely. If our goal was to purely increase traffic flow, an underpass may have been a better solution.

Once we have a good idea of the rules we think we should be following, we need to create a strategy that will ensure all the people interacting with the system will understand these rules and guidelines. Think how much effort is spent training new drivers. We may find this annoying when we go through it, but ask the average driver if they would prefer that people ‘learn by getting out on the road’ and I think most would think this a terrible idea. There is a reason we have drivers licenses. In the SharePoint project this translates to training. We need to ensure that our users are shown what hey should and shouldn’t do. This includes explaining our general purpose for the existence of the system as well as the specifics of how to perform certain actions. If you only know how to drive the car, you are not yet ready to get out on the road.

Lastly, we need to know how well our system is doing. One of the most famous and successful traffic solutions on the planet are the German motorways, known throughout the world as ‘the Autobahn’. A key ingredient to this complex system is monitoring. There is an entire team constantly monitoring the situation of all the roads in real-time and adjusting the speed conditions etc. to ensure most optimal operation. Our SharePoint projects will generally not have the luxury of such monitoring, but that doesn’t mean we can forget it all together. We need to define measures of success, and keep an eye on our SharePoint solution to see how it is performing. This includes measuring technical data such as bits and bytes but also more human data such as user satisfaction. We can then react to our measurements and tweak the system or the rules and guidelines around the system to improve the overall performance.

So what about Governance? We’ve somehow lost that word in the last few paragraphs. Governance is a collection all the rules and guidelines that get our project from the initial vision to a operating and continuously adapting solution. Governance is everything from the hard rules of how often we perform a backup to the soft guidelines that let our users know which content type they might want to use for a given document. It also includes the rules of the project itself, such as what type of user training do we perform, and how do we measure success. There are many facets to Governance in a SharePoint project and each of them is crucial to the overall success.

I hope you now have some understanding of what this Governance thing is, and why you should care about it regardless of what your role on the SharePoint team. It comes down to you and your team, do you want to build the autobahn, or are you ok with spending a lot of time, money and effort on this:

PLUG: I have to give credit to the SharePoint Information Architecture and Governance Master class since the idea for this blog post originates from discussions during that class and I would not have written this if it wasn’t for that fantastic course. If you want to know more about this topic, I highly recommend attending.

Also thanks to 21 Apps and the work they are doing on Governance in the SharePoint space, awesome work guys, keep it up.

Navigation on Public Facing SharePoint Sites

One thing that all sites (SharePoint or not) need is some form of navigation. This is true for all sites, not just public sites. There are plenty of types of navigation elements, but if we stick to public sites, there tend to be the following few types:

1. Main navigation. This is almost always a horizontal list of links across the top of the page. These links do not change regardless of where on the site the user is. Sometimes each link will have a sub-navigation that is shown using JavaScript when the user hovers over a link.
   
2. Contextual navigation. This navigation can manifest itself in many ways, sometimes as a tree view, sometimes as a flat list. In any case, the content of this navigation is usually different per page, or at least per section of the site.
   
3. Reverse Navigation (Bread crumb). This is some type of path allowing the user to see where in the site they are, and to navigate back the way they came from.
   

In SharePoint, we have out of the box elements that provide all three types of navigation mentioned. They may not work exactly how we need them, but they are there for the most part.

On projects I was involved in, we never ended up using any of the built in navigation controls. Even thought they have been improved for SharePoint 2010, they still render html that never happened to fit the html we needed for the site. So how did we solve our navigation needs? There are a few options that I have used, I’m sure there are others out there:

1. Static, hard coded links. This scares the crap out of some people, but I argue there are still plenty of scenarios where hard coded navigation is perfectly acceptable. As an example, take the apple.com website. How often do those links across the top change? Almost never. And when they do change, it is likely due to a very rigorous change management process. This is a case where I would advocate a hard coded solution. On the rare occasion the links change, update the master page.
   
   Advantage: Best performance, Easy
   
   Disadvantage: Potential for dead links
   
   
2. Custom navigation controls based on SharePoint navigation providers. Just because the web controls that SharePoint delivers may not suit your needs, the navigation information they contain may be just what you want. In this case, use one of the providers that SharePoint offers. See MSDN on how to get started. http://msdn.microsoft.com/en-us/library/bb897657.aspx
   
   Advantage: Dynamic, Flexible
   
   Disadvantage: Can be hard to get right, Usually lots of work
   
   
3. Navigation that has nothing to do with navigation. What do I mean? Show a list of the published pages from the pages library of the current site. That has served us as a viable navigation strategy on a number of sites. Usually best suited for the contextual navigation category. There is a strong potential downside of using this approach. It is very simplistic and doesn’t implement any performance optimizations or security trimming. So if used in an incorrect scenario, it can have nasty side effects. For me this method has generally been useful when there are a number of content pages that need to be shown for a certain category, etc, and if a page was published, it should be visible for everyone.
   
   Advantage: Relatively easy to build, Dynamic
   
   Disadvantage: Only suitable for simple cases
   
   
4. Search. Yes, you can use search as a viable navigation method. There are ways that you can prepopulate a search result set and display this as navigation. We used this on a project for a “Related Items” type of navigation.
   
   Advantage: Very dynamic
   
   Disadvantage: Depends on good search results, Can’t always know what will be in there
   

So as usual with SharePoint the right answer for whatever you are doing is “it depends”. I would however stress that whatever your choice of navigation solution for a public site, don’t forget that you are building for the end user and make sure you are sending friendly HTML, etc to the client.

Build for the end user

I attended a session at the European SharePoint conference today about building public facing websites using SharePoint. I respect the effort that the speakers put into their session and I agree that they had a nice looking web site created. However I strongly disagree with the methods they demonstrated and was disappointed to see these methods demonstrated as good practices.

There were two specific methods that I strongly disagree with demonstrated at the session:

1. 
   Hiding the ribbon for anonymous users with CSS.
2. 
   Using web part zones and web parts like the Content Query Web Part on public facing page layouts.

Both of these techniques are a symptom of what I call building for the editor. As developers we often work very closely with the editors when developing the site, and we thus tend to focus on making the site in such a way that the editor has a very easy life. This is not necessarily bad, but it comes at a cost. If we look at the big picture, the ultimate reason for the website being built is to be consumed by the end users. Editors are an important part of the picture, but we should be building the site for the end users. So why would that be different than building for  editors? Simple answer: page weight.

Why care about page weight? The days of large bandwidth and low latency are over (if they were ever here). Many users these days are accessing your site via networks that are not gigabit LAN, but instead are  3G networks or something similar and will notice when your home page is 300kb and sends another 300kb of JavaScript and CSS along. There are real costs attached to this in money, battery life and time spent waiting.

The good news is that there is a solution. In a previous blog post http://jcapka.blogspot.com/2011/08/public-facing-sites-using-sharepoint.html I introduce a webtemplate that includes a very lean masterpage and serves as a good starting point for building public SharePoint sites.

There are comments and then there are comments

The title may be somewhat vague, but in this post I want to alert you to the different type of comments you can use when working with ASP.NET. I have seen too many developers not understand the difference between

<!-- Comment-->

and

<%-- Comment --%>

The first type is an HTML comment which instructs the browser to ignore the contents. The second type is  an ASP.NET comment which instructs the ASP.NET parser to ignore the contents.

This is a very important distinction. The first type of comment means nothing to the ASP.NET parser, and is handled just like any other HTML in the page. That means that if I put something along the lines of:

<!-- <asp:Literal runat="server" Text="joe was here"/> -->

into my web page, the ASP.NET parser will stil parse this and execute the code in that literal. So my HTML will look something like

<!-- joe was here -->

When this becomes very important is at times where a developer thinks they have commented out some controls, but they haven't really, it's just that the output of that control isn't being displayed by the browser.

Imagine a line like:

<!-- <myControls:SomeVerySlowThing runat="server"/> -->

A novice developer may be under the impression that 'SomeVerySlowThing' has been commented out, but in reality, the code for that control is still being run.

Another time this is important is when trying to create lean and efficient HTML. Adding comments to HTML is handy, it is however still content being sent down the wire. If you are trying to make your HTML as lean as possible, use the ASP.NET comments and they will not show up in the end result HTML.

Free Brainstorming Session

Are you interested in what SharePoint is and what it could do for your business? XComplica is currently offering a free brainstorming session to help you answer these questions!

Visit www.xcomplica.com/Promo for more details.

Public facing sites using SharePoint 2010

This post is the first in a series that will walk you through building public facing sites in SharePoint 2010. I will discuss a method that I use for this process, whether or not it is the best method I leave up to you to decide.
In the past I used site definitions, but I recently decided to switch to the new web template method. In order to save myself and others time in the future, I have created a starter web template for public sites. You can download this at Codeplex: http://xcomplicaps.codeplex.com

I have to thank Mirjam van Olst and Vesa “vesku” Juvonen for their blog posts that were extremely helpful in putting this web template together.

http://www.sharepointchick.com/archive/2011/02/10/using-web-templates-to-create-site-collections.aspx

http://blogs.msdn.com/b/vesku/archive/2010/10/14/sharepoint-2010-and-web-templates.aspx

In this post, I will walk through the components in this web template, and how you can get started using it. Later posts will build on this one.

If you haven’t already, you may want to download the solution from Codeplex, and open it in Visual Studio.
Disclaimer: The solution is meant to be used on a development machine running SharePoint 2010. I also have the CKS Development tools installed and urge you to do the same.

You will see that it is a relatively small and simple solution containing one project. The project consists of a number of components, and we will go through each one.

Web Template (XComplicaPublicStarter)
There are two files that define the web template. There is the onet.xml file and the Elements.xml file. The Elements.xml file tells SharePoint about the web template and the onet.xml file defines the components of the web template. This means that any changes to the information about the web template, such as its name, description and display category should be changed in the element.xml file. This is similar to the webtemp_zzz.xml file for site definitions. The onet.xml file is a very stripped down version of the onet.xml used by the Publishing Portal site definition (BLANKINTERNET#53). Everything has been removed except for one configuration, and this configuration contains nothing except site level features and web level features. The features are the same as for the Publishing Portal, except there is one extra added at site scope and web scope, and these add the custom components we will be discussing later in the article. Two changes have been made to the web scoped publishing feature. The master page has been changed to a custom master page, and simple publishing has been turned on.

Three Features
There are three features as part of this project, one for farm scope, one for site scope and one for web scope.

The farm scoped feature has only one component, and that is the web template just discussed. It ensures that you can create a site collection based on this web template.

The site scoped feature contains elements that are global to the site collection as listed below. These will each be discussed in detail later:

• Master pages
• Page layouts
• Site columns
• Content types
• Global branding files

The web scoped feature contains elements that apply to the web itself and these will also be discussed in detail later:

• Content type binding
• Home Page
• Property Bag Setter

The web scoped feature also includes an event receiver. This event receiver enables anonymous access to the web. Since this is a public facing site, it is handy to have anonymous access automatically enabled on deployment.

Master pages
In the modules section, you will find a module called master pages. There are two master pages here, one is the very well known starter master page by Randy Drisgill, _starter_publishing.master.

http://blog.drisgill.com/2010/02/microsofts-sharepoint-2010-starter.html

The other master page, xcps_blank.master, is a stripped down version of Randy’s master page. This master page is what I use as a starter master page, and it has very little visible content. In this page I have commented out or removed as much of the html as possible, and I placed those placeholders that I could not remove into hidden panels. This technique is nothing new, I just took it further than others. There are only two content placeholders that are left in the visible portion of the page, the PlaceHolderMain and the PlaceHolderPageTitle. I assume these will be used on most if not all public facing sites.

I have also added a few links to content in the head section of the master page, since I assume every site these days will include at least one css file, one javascript and likely a favicon. These three branding files are located in the branding module and are meant to be changed when creating a new site.

I also add a reference to jQuery, but note that this is a reference to the CDN version of jQuery. There is no need to host this file locally.

As you develop your site, you will likely want to use one of the other placeholders or other components that are currently hidden. This is perfectly fine, just cut and paste them from the hidden panels into the appropriate location.

Note that I have also removed the ASP.NET form tag. I don’t see a need for it in a clean public facing site, but if you do find yourself needing it, feel free to add it back. There is a great article on how to make web parts work without the form tag on Waldek’s blog.

http://blog.mastykarz.nl/web-parts-in-content-with-master-pages-without-the-form-tag-no-problem/

Page Layouts
There is currently only one page layout in the project, and that is for the home page. It is very simple since this layout will differ for every site. I have added some very basic components in the PlaceHolderMain just to demonstrate how content can be rendered. There is a standard SharePoint Webcontrol for rendering html and there is a custom control that I developed for rendering Publishing Images. SharePoint provides a control called RichImageField http://msdn.microsoft.com/en-us/library/microsoft.sharepoint.publishing.webcontrols.richimagefield.aspx that is meant for this, but I find it renders too much extra html around the image. I therefore created a simple server side control that renders just the img tag and some core attributes. You can see (and change) the code for this in the ServerControls folder of the project.

Site Columns (Fields)
Most public facing sites will contain at least a few different types of pages. These different types of pages should be represented by a number of content types, and content types are composed of fields. The StarterFields element contains one file which defines three site columns. One is a simple numeric site column used for ordering pages and the other two are Publishing site columns used to hold the html and image that the home page renders. You will add fields to this file, or create a separate element for your fields as you need them.

Content Types
Two content types are defined in the project. The generic content type is meant as a base type for all pages. It inherits from the built-in Page content type and it contains any fields that are common to all pages in the site. As it is, it contains one field which is the Sort Order. I find that a common request from authors somewhere down the line is to be able to order pages in some custom order, and so I always ensure this field is in all page content types.

The other content type is for the home page. It inherits from the generic content type and contains two fields as a sample. These are the Main Image and Body Text fields. Any site you build from this template will likely have different needs, so this will change. Note that I strongly advise using a separate content type and page layout for the home page and not share these with other pages. The home page of any site tends to be a very special page that evolves and changes in a different fashion from the rest of the site.

Global branding files
Every public facing site will need some custom CSS, JavaScript and probably a favicon. I have created a module to contain these files, and they are deployed into the Style Library. There are a number of discussions out there regarding the use of the Style Library, read Andrew Connell’s article for a good introduction.

http://msdn.microsoft.com/en-us/library/dd221375.aspx

Note that I have referenced the CSS, JavaScript and favicon files in the master page.

Content Type binding
Every web that has the publishing features enabled has a Pages library. If we want to be able to create pages based on the content types we define, we need to bind these content types to that Pages library. There is an element file in the Content Types folder that takes care of this, just remember to add a reference in that file to any new content types that you create.

Home Page
The home page is an instantiation of the home page layout, the home page content type and some actual content. The home page module is where this is all put together. The aspx page itself just has a reference telling SharePoint to use a page layout, this is the same for every instance of a publishing page. The interesting stuff is in the elements file, which defines the properties of the page, as well as the location (Pages List) that the page should be deployed to. Notice that the properties include the page layout, the content type, but also properties like Title, Body Text, etc. Also note that the MainImage property has been specified and contains a link to an image from teh style library. That image is provisioned as part of the branding module. In a different post I will show how to provision actual content into image libraries. For most pages we would allow authors to add the content, but for some pages as well as for test purposes it can be useful to prefill some content in.

Property Bag Setter
Due to the way web templates work, there is no way to know what web template was applied once the provisioning of the site is finished. It is therefore best practice to add a property in the web’s property bag in order to store this information. See the links above about web templates for more info on this.

Getting it all working
Now that you’ve had a full tour of the project, you should be able to create a site based on this template. You first need to specify a Site URL in the project properties so that it points to a valid SharePoint URL. What is somewhat annoying here is that we will be creating a new site collection based on this web template, but there already needs to be a site collection at the Site URL you specify. I often just start by creating a blank site at that URL, since I know I will be deleting it right after deployment. There are some PowerShell shortcuts you can also take, but I’ll leave those for another time.

Once your project has deployed, go to central administration and open up the page for creating site collections. You should see a new tab, and in that tab the Public Starter template.

Before you create a new site collection, I suggest you delete the blank site that you needed to deploy the project. This is up to you, but if you do so, you can create your new site collection at the root URL.
From this point, it is the same as creating any out-of-the box site collection. Once the process completes, you should be greeted with a new site based on the components discussed. Just for fun, look at the source and the page weight of your new site, and compare this to the regular SharePoint publishing template.

Conclusion
In this article I walked through the different pieces that make up the XComplica Public Starter web template. The purpose of the template is to create a clean starting point for creating lean and clean HTML sites using SharePoint. In future articles I will expand on how to create different types of pages, how to implement custom html into this template and other topics.