Tag Archives: Developer

Search Schema Scoping in SharePoint Online

For solutions that are contained in a single site collection, or span a small number of site collections, or are in a tenant where the other solutions are not trusted or are unknown, then I have a strong preference to use site collection scoped search schema rather than tenant scoped.

Side note: I am yet to come across a situation where I would use site scoped search schema. In my mind, the existence of search schema at this level only serves to confuse.

Search Schema hierarchy is SharePoint Online.
Search Schema hierarchy is SharePoint Online. There is also site scoped search schema at lowest level which is not present here.

For those that aren’t fully aware, search schema (the set of managed properties that are accessible via the search framework) can be provisioned at the tenant, site collection, or site scope. These scopes are hierarchical such that managed properties are inherited from¬†the tenant scope down to the site scope but can be overridden along the way. There are some good articles that delve into this in more detail.

By provisioning search schema at the site collection level you are mitigating the risks of errors related to other solutions changing the properties which your solution relies upon. This is especially relevant in SharePoint Online where all solutions in the tenant have to share a common set of RefinableTypeXX managed properties.

There are some important exceptions, of course.

  1. People Search, a.k.a User Profile Search, a.k.a Local People Results
    In SharePoint Online, people properties are indexed on a very slow schedule. We requested more information from Microsoft regarding this and were told that this schedule is ‘confidential’. I have found that when using site-collection scoped managed properties it can take *weeks* for them to get populated. I have found much better (although still poor) performance using tenant scoped properties (usually within a few days). Assuming you do require custom search schema for people properties I would still recommend provisioning all remaining managed properties (all those not mapped to people properties) at the site collection level.
  2. Many site collections
    Of course, having many site collections which require the same search schema is valid reason to go tenant scoped. This is purely due to management of the properties going forwards. A solid scripted deployment procedure should not care if you are provision search schema to 1 or 50 site collections – but anyone maintaining the solution will definitely care if they have update 50 schemas manually, or are suddenly required to script something which they feel should be *easy*. Even in this scenario you should still consider how much you trust other solutions in the tenant against the impact of finding out that one day your managed properties are mapped incorrectly. Depending on your solution this could lead to errors that are left undetected, or conversely obviously break your home page.

Paul.

Follow documents from search hover panel

There is a somewhat confusing logic behind when the FOLLOW button is displayed on the search results hover panel (a.k.a document preview).

A document hover panel with both the POST and FOLLOW buttons present
A document hover panel with both the POST and FOLLOW buttons present

What I am talking about?

If you are building a solution that relies on the following of documents but you are using Yammer rather than the SharePoint social feed then you may be wondering why, from the search results hover panel, you can follow pages, users, sites, but not most document types.

NB. If you are finding that you can’t following anything, check that web scoped feature ‘Follow Content’ has been activated on each site which contains content you wish to be able to follow.

NB. You can still follow the document types in question by clicking ‘view in library’ and using the library item menu to follow.

In many cases, wanting both POST and FOLLOW doesn’t make a lot sense as a primary reason of following documents is to populate the activity feed which is not available when Yammer is being used as the enterprise social experience. As such, please consider why you want this behaviour at all. In my scenario the user’s list of followed documents is promoted to the home page and bookmarking documents is a key user story.

What is going on?

The search results hover panel is built from a number of display templates which you can read about in more depth here (TechNet) or here (Chris O’Brien) or many other places.

Importantly, there is a display template which defines the common actions (buttons) across the bottom of the hover panel and when to display them. The display template is called Item_CommonHoverPanel_Actions and can be found here:

Site Settings > Master Pages and Page Layouts > Display Templates > Search > Item_CommonHoverPanel_Actions.html

If you inspect this display template you will find an if else block around the rendering of the POST and FOLLOW buttons. The logic can be summarised as:
The POST button is visible if Yammer is enabled and the result type supports it, otherwise the FOLLOW button is visible if the result type supports it, at no time will both buttons be visible.

If you download a copy of the display template HTML file, update it to remove the ‘else’ as in the code snippet below, and then upload it again, you will find the both the POST and FOLLOW buttons will be displayed in the search hover panel when supported. Success!

But is it okay update that file?

The short answer is yes. Take care as this file is used by every hover panel in SharePoint (to my knowledge, there may be some completely unique ones) and so changes could break something that isn’t obvious.

The major risk is that if Microsoft decide to update the hover panel which require them to produce a new version of the display template file (they have done this previously when introducing the POST button). In the case that you have modified this file, then your changes will be lost. This can happen without warning (unless you have a second tenant on first release to catch these issues before they hit production – you should be doing this!).
For very minor updates such as this, and to support non-critical functionality, it may be okay to make these changes and be prepared to re-implement them should Microsoft issue an update.

The alternative is to make a copy of the display template with a new name. This approach means that your changes will not get overridden but it also means that your solution will not get the updates that would otherwise be pushed to this file. We call this ‘customisation tax’ and it is a trade off as to which way you’d rather push changes.
In this particular scenario this latter approach is not very practical as every result type references the existing display template. You would be required to make copies of all the result type display templates that are applicable (possibly a dozen or more), and update the result types themselves to use your new templates. Unless you are bypassing result types and using a single display template for all results, this feels overly complex for such a minor change, but major changes will necessitate the effort.

EDIT: A colleague of mine, Luis Manez, pointed out that with a little JS you can force a custom hover panel to be rendered for all result types. You can read about it (approach one) and some other approaches to associating custom hover panels here (Elio Struyf).

Paul.

Increase Search Result Limit Beyond 50

Content Search Web Parts (CSWPs) and Search Result Web Parts (SRWPs, a.k.a Result Script Web Parts RSWPs) prevent more than 50 results being returned at once. This is true with or without paging enabled. This is true even if you provision the web part using XML. In older versions of SharePoint, it may allow you to enter a row limit greater than 50 but will then default to a mere 10 results.

increaseSearchResultsRowlimit

I’m sure Microsoft has very good UX and technical reasons behind this limitation, however at 100 items I don’t feel that either UX nor performance suffer. I imagine that if using this technique for returning large page sizes (>100) that there is very real potential for bad performance and the UX is most likely appalling.

Importantly – this code should only be considered a learning tool. This code is entirely unsupported and generally just a really bad idea. Please be responsible ūüôā

Recognition to Matt Stark who provided this solution. All I have done is rewritten it a bit for safety and I wanted to prefix it with a bit more discussion than it had. Original discussion is here.

The web parts which you want to act upon must be set to load asynchronously (this is not the default mode).
Edit Web Part > Change Query > Settings > Loading Behaviour :: Async option

Edit Web Part > Change Query > Settings > Loading Behaviour :: Async option
Edit Web Part > Change Query > Settings > Loading Behaviour :: Async option

Take note:

  1. There are a few available solutions to this, none of which are much good, and even this one is should be considered a hack and be treated with caution.
  2. I will only discuss this option as it is the best I have come across when considering the limitations of SharePoint Online.
  3. Please only include this code on specific pages using a Content Editor or a Script Editor, and NOT globally.
  4. I have amended the original code such that it only changes the row limit for those web parts which are set to return 50 items exactly.
  5. I haven’t found any issues with this code, however I am using on a page that does nothing but display a list and in a situation where failure has limited impact on the system.
  6. Apparently doesn’t work for anonymous users

And finally the code…

// <script type="text/javascript">
var CC = window.CC || {};
CC.CORE = CC.CORE || {};

CC.CORE.IncreaseSearchResultsMax = (function () {
    "use strict";

    var $ocreate = null;
    var newMaxItems = 100;
    var oldMaxItems = 50; // web part must be set to show this many items

    // on application initialization 
    // steal the global create variable and 
    // intercept calls to create UI widgets.
    Sys.Application.add_init(function() {
        $ocreate = $create;
        $create = updateResultCountCreate;
    });

    // listen to UI widget calls for CBS & DP
    var updateResultCountCreate = function (a,b){
        var ps = Array.prototype.slice.call(arguments, 0);  
        if(a === Srch.ContentBySearch && b.numberOfItems === oldMaxItems) { 
            b.numberOfItems = newMaxItems;
        }
        if(a === Srch.DataProvider && b.resultsPerPage === oldMaxItems) {
            b.resultsPerPage = newMaxItems;
        }
        $ocreate.apply(this,ps);  // apply the original $create method that we stole
    };
    return true;
})();
// </script>

Paul.

SPO CSOM Error: For security reasons DTD is prohibited in this XML document

I found myself encountering the following error when authenticating to SharePoint Online using CSOM from PowerShell:

DTD_error
Exception calling “ExecuteQuery” with “0” argument(s): “For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.”

I believe that there a number of causes for this issue some of which are firewall and ISP related. This may only resolve a subset of the cases where this issue has been arising, even under the same circumstances.

In my scenario, I found that this issue was only arising when the credentials I was passing were being federated. That is, when the username was *not* in form <me>@<domain>.onmicrosoft.com but rather something like <me>@<domain>.co.uk. It is also possible that this issue resolves itself after a single successful authentication has occurred. Try providing credentials for a *.onmicrosoft.com account, and if that works try again with a federated account. This is discussed more later.

I used Fiddler to compare the request/response trace from a successful authentication and one where this error occurs. It turns out that somewhere internally a request is made to msoid.<full-domain> where <full-domain> is the bit after the @ symbol from the username provided. In the case where this value is of the *.onmicrosoft.com variety, a 502 error (no DNS entry)¬†is returned with no request body and the authentication proceeds successfully. In the other case, the ‘msoid’ URL is resolved and a response with a request body is returned.
In my case the response was a 301 error (permanent relocation), however I read of cases where a 200 (success) has been received. Importantly to note, is that the response, success or otherwise, returns an HTML body containing a DTD (Document Type Declaration), and in turn produces the rather unhelpful error message.

So how do you fix it? Well one way is to provide an entry in your hosts file which ensure that the msoid URL will be invalid. I found that providing a local host entry for it worked. Your hosts file can be found here:
C:\Windows\System32\drivers\etc

I added a line which looks like the following:

127.0.0.1        msoid.<domain>.co.uk

And it worked! Intriguingly I found that if I then removed this line from my hosts file, SharePoint Online authentication from PowerShell continued to work. It is for this reason that I suggested trying to use a *.onmicrosoft.com account first at the begging of this post Рjust in case it resolves the issue for you without touching the hosts file. Please comment if you have any success (or otherwise) with that approach.

Hope this helps! Good luck.

SharePoint licencing limitations: Standard vs Enterpise Features and Kiosk Users

When discussing Office 365 licencing there are a number of things that as an architect or developer you must be aware of.

EDIT: A handy Excel tool for checking which features are available for different licence types can be found here: Office 365 service comparison

Image of Buying kiosk licences
Buying kiosk licences

Creating solutions for limited users
Kiosk users are the cheap users will the greatest restrictions. However the limitations placed on these users really are quite manageable in many circumstances and shouldn’t cause you particular worry when developing a solution for these users. The key points to remember when providing a solution to these users is:

  • They don’t have a user profile. They can still view the ‘My Settings’ page, but not the ‘About Me’ page. These users still have the full set of user profile properties which can be set by an administrator or via AD synch and programmed against.
  • They can only use Office Web Apps in READ mode. They cannot edit documents with a client version of the correct Office application. Kiosk users from a K2 licence (opposed to K1) can also edit documents using OWA.
  • They can’t be administrators at the tenant or site collection level. However they can be granted Full Control permissions.

Be aware of the feature set available in Production
Many features will not be present or will not work under some licencing schemes. The primary issue which I have encountered is around content rollup. Licences which do not support the Enterprise feature set do not support the Content Search Web Part. You can use the Results Script Web Part instead, but remember that the display templates used are not transferable. The Content Search Web Part display templates reference the Srch javascript namespace which will not be present if using the Results Script Web Part.
There are obviously many other Enterprise features which I won’t mention explicitly but have a browse over the below table:

 

Developer features Only in SharePoint Server 2013‚ÄĒEnterprise Edition
Access Services Yes
BCS: Rich Client Integration Yes
BCS: Tenant-level external data log Yes
Custom Site Provisioning Yes
InfoPath Forms Services Yes
Analytics Platform Yes
Improved Self-Service Site Creation Yes
eDiscovery Yes
Preservation hold library Yes
Video Search Yes
WCM: Catalog Yes
WCM: Cross-site publishing Yes
WCM: Faceted navigation Yes
WCM: Image Renditions Yes
WCM: Multiple Domains Yes
WCM: Topic Pages Yes
Business Intelligence Center Yes
Calculated Measures and Members Yes
Data Connection Library Yes
Decoupled PivotTables and PivotCharts Yes
Excel Services Yes
Field list and Field Support Yes
Filter Enhancements Yes
Filter Search Yes
PerformancePoint Services Yes
PerformancePoint Services (PPS) Dashboard Migration Yes
Power View Yes
PowerPivot Yes
Quick Explore Yes
Scorecards & Dashboards Yes
Timeline Slicer Yes
Visio Services Yes
Content Search Web Part Yes
Custom entity extraction Yes
Extensible content processing Yes
Query rules‚ÄĒadvanced actions Yes
Search vertical: ‚ÄúVideo‚ÄĚ Yes
Tunable Relevancy Yes

Setting Default Page Layout, Available Page Layouts, and Available Web Templates via CSOM (JSOM)

The publishing infrastructure provides a site with many features including page layouts. A default page layout, a restricted set of available page layouts, and a restricted set of web templates can be set. The page in the UI looks like this:

config_pageLayouts

In order to perform these configurations programmatically, methods are provided as part of the the server side object model via the PublishingWeb object. However setting these properties via CSOM is less obvious. The equivalent methods are not available (at least not at the time of writing. It is worth noting that the CSOM model gets extended as part of CUs as well as in SPO).
The property information is stored solely as web properties. I have verified this by reflecting the Microsoft.SharePoint.Publishing assembly, and ensured that the server side object model methods perform no other action than this.

The following JSOM code demonstrates how to set the web properties with valid XML values.

The methods can be called like this:

// The page layout is defined by its filename
var filename = "NewsArticlePage.aspx";
setDefaultPageLayout(filename, onSuccess, onFailure);


// The page layouts are defined by an array of filenames
var filenames = ["NewsArticlePage.aspx"];
setAvailablePageLayouts(filenames, onSuccess, onFailure);

// The web templates are defined by their long name,
//the guid is that of the feature which deploys the web template
var webTemplates = ["{BDACFFF5-05DF-4446-9907-B4C39F15F1D7}#WT_VanitySite"];
setAvailableWebTemplates(webTemplates, onSuccess, onFailure);

 Note Рthese samples reference both the jQuery and underscore libraries.

Deploying/provisioning difficult web parts (like XsltListViewWebPart)

Deploying the XsltListViewWebPart to a page as part of a packaged solution can be a challenge (as well as some other web parts). When these web parts are exported they refer to the underlying list or library via its GUID rather than by URL or name. This means that that the web part cannot be imported using this same XML to another site as it will fail to find a list with that GUID.

As Microsoft’s guidance is now to avoid using the server-side object model, the previously most common way to resolve this issue¬†is no longer available. Web parts could be provisioned by JSOM code¬†on a page however that really¬†feels messy and there some declarative alternatives.

SharePoint-2013-export-webpart

Replacing the AllUsersWebPart element with the View element

Specifically for provisioning an XsltListViewWebPart, rather than using the AllUsersWebPart element to provision a web part to page you can use a View element instead. The view element allows you to specific a list via URL, which when omitting a list reference in the child web part element, will be used to identify the list or library. This is my recommended approach. See here for the View element schema definition.

 

Using the listId token for document libraries

There is a token that can be used to replace the library GUIDs in the web part XML. I say library specifically as I can’t seem to get this approach to work with lists. The properties that need to be replaced with token are as follows (in this example we are referencing the ‘Documents’ library):

<property name="ListName" type="string">{$ListId:Documents;}</property>

<property name="ListId" type="System.Guid, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089">$ListId:Documents;</property>

An example of the full XML using this approach can be found here.

This method can be used for other web parts as well such as specifying a specific list when deploying a ContentQueryWebPart.

Using BinarySerializedWebPart for lists or libaries

The other option is a very static and rather awkward approach to solving this issue – but at least it works. The BinarySerializedWebPart allows any web part to be imported in a binary format and provides a mechanism for mapping GUID to URL. The major downside is that if you want to make a minor change it will require you to re-create the binary representation of the web part.

In order to find this XML you will need to configure a web part as desired on a site that has never had publishing features enabled and then import the site as a site template (the publishing features disable the export site template functionality and although you can get around this you can get very strange side-effects, so I recommend avoiding doing so. e.g. I once found that all my Script Editor web parts were deleted from my site after performing the operation on site with publishing features enabled). By downloading and renaming the site template package as a .cab you will be able to find an Elements.xml that contains all the xml for deploying web parts, including yours as BinarySerializedWebPart if it can’t be otherwise imported.

For more information on this see here.

Good luck!

Key difficulties deploying a SharePoint Online solution using CSOM

I have been developing a console app that utilises the SharePoint C# CSOM to deploy a solution to SharePoint Online (a.k.a Office 365 SharePoint). The solution involves more than just a wsp (although it has one of those too). I have encountered a few difficulties during this process and this blog will discuss those:

  • (Re)creating a site collection
  • Importing a large-ish taxonomy
  • Uploading and installing a sandboxed solution (that contain only declarative elements)
  • Hooking up¬†of taxonomy and (root site) lookup columns
  • Pre-creating a number of sites with specific features enabled (including the root site)

Before I go any further, for those of you reading this before doing something similar yourselves, please be aware of two constraints which caught me by surprise:

  • You can’t leverage the same import taxonomy function that is available in Term store management. If you¬†already have files in that format you will need some custom code (I have an example later on) or you may want to import from a more robust XML formatted document
  • The CSOM does not support uploading or activating sandboxed solutions! However, there is a CodePlex project that assists with this. I also include the dll later in the post that I have rebuilt with references to the lastest v16 Microsoft.SharePoint.Client dlls.
  • The CSOM does not support activating web scoped features! You can active site scoped but not web scoped. You need to use web templates to achieve this. Again, I will cover this in some more depth later on.

SharePointOnline2L-1

Deleting and recreating a site collection

The initial step of the deployment process involves creating a new site collection (having deleted it first as required). In order to perform actions at this scope (tenant) you cannot create your client context in the same manner as usual (with a reference to the site collection; as it is yet to exist and the site collection delete and empty recycle bin require it too). Instead you must create the client context passing in tenant/admin site URL.
This is the one that looks like this: https://<tenant>-admin.sharepoint.com

You can then create a Microsoft.Online.SharePoint.Tenant object by passing the ‘admin’ client content to its constructor. This object requires a reference to the Microsoft.Online.SharePoint.Client.Tenant assembly which is available¬†by downloading and¬†installing¬†the SharePoint Server 2013 Client Components SDK. The assembly can then be found here: C:\Program Files\SharePoint Client Components\16.0\Assemblies

The tenant object provides the methods required to perform the create and delete site collection actions. This process involves a lot of waiting about for deletion to complete, and then provisioning to complete. Unfortunately you can’t continue with other actions until this has occurred. I found this to take upwards of three minutes.

A link to the relevant code that I used to achieve this can be found here: https://gist.github.com/paulryan/cbfaa966571d6a9cdb8b

Importing taxonomy

As mentioned above you can’t pass those CSV files directly to the CSOM and have it import it all for you. In my scenario we had already developed a lot (dozens) of term sets in the form of these CSV files so that¬†were able to import them during a discovery phase so it was important that I could support the import of¬†taxonomy in this form. I wrote code to support in the import of these files, but only to the point that it meets my immediate requirements. Please use the following as a rough guide only¬†as it is not fully featured (or tested beyond the happy path).

The code I wrote to support his can be found here: https://gist.github.com/paulryan/e461f8bac28336b05109#file-importtaxonomycsom-cs

Uploading and activating a sandboxed solution

There is a¬†CodePlex project that provides this functionality (as well as some authentication utilities) that I mentioned above. It¬†performs web requests to UI¬†and I am very¬†glad someone else has already done¬†this¬†before me! It was originally created when SharePoint 2010 was present in the cloud and references the v14 assemblies of the Microsoft.SharePoint.Client assemblies accordingly. If you don’t mind maintain references to both v14 and v16 assemblies then this might be fine. I have instead rebuilt the source having replaced the references with the v16 equivalents.

You can download it here: SharePointOnline.Helper.dll

FYI: v14 is SharePoint 2010, v15 is SharePoint 2013, v16 is SharePoint 2013 Online specific

Activating web features

Actually there isn’t a lot more to say here other than you must use web templates if you need to create sites with features enabled as part of the deployment process as it can’t (currently) be done using the CSOM. I would recommend using the web template for nothing other than activating features and put all other declarative elements in a feature. This will provide the best upgrade experience in the future.

Hooking up taxonomy columns

The best place to start is almost certainly a reference to Chris O’Brien’s blog on this here. As I have the luxury of being able to run further deployment code after uploading/activating the sandboxed solution I opted to avoid having to rebuild the solution for various environments and instead hook-up the columns using the CSOM and a mapping. There is a catch with this though.

If your list instance is built from a list template which defines the managed metadata columns then updating the site column via the CSOM fails to push down the new SspID. To get around this, DO NOT include managed metadata column definitions as part of the list definition (in the fields element). When you run the CSOM to update the site columns it will update the content type and add the column to the list instance with the correct SspID.

Good luck building your SharePoint Online CSOM deployment framework!

Script Editor only runs JavaScript in edit mode

If you need to embed script into a content editable page in SharePoint 2013/Online, you may decide to use the new Script Editor web part. There are often many preferable ways to add script to a page (e.g. via the master page, a custom action, custom control, the ScriptLink property, etc.) however this is an easy option for demo purposes or when deployment activities are out of scope.

There is a gotcha for those who like to skip attributes that may have seems verbose in the past. Any JavaScript which you include via the Script Editor web part must be wrapped in the <script> tag otherwise it will be rendered as text. However, if you fail to provide the required type='text/javascript' or language='javascript' attribute to the script tag then the code will continue to run when the page is in edit mode but will fail to execute when the page is saved and then viewed.

Note the attribute, you need it!
Note the language attribute, you need it!

CAVEAT: As I stated in the first paragraph, this is often not the best way to add script to a page.