Tag Archives: SharePoint

Follow documents from search hover panel

There is a somewhat confusing logic behind when the FOLLOW button is displayed on the search results hover panel (a.k.a document preview).

A document hover panel with both the POST and FOLLOW buttons present
A document hover panel with both the POST and FOLLOW buttons present

What I am talking about?

If you are building a solution that relies on the following of documents but you are using Yammer rather than the SharePoint social feed then you may be wondering why, from the search results hover panel, you can follow pages, users, sites, but not most document types.

NB. If you are finding that you can’t following anything, check that web scoped feature ‘Follow Content’ has been activated on each site which contains content you wish to be able to follow.

NB. You can still follow the document types in question by clicking ‘view in library’ and using the library item menu to follow.

In many cases, wanting both POST and FOLLOW doesn’t make a lot sense as a primary reason of following documents is to populate the activity feed which is not available when Yammer is being used as the enterprise social experience. As such, please consider why you want this behaviour at all. In my scenario the user’s list of followed documents is promoted to the home page and bookmarking documents is a key user story.

What is going on?

The search results hover panel is built from a number of display templates which you can read about in more depth here (TechNet) or here (Chris O’Brien) or many other places.

Importantly, there is a display template which defines the common actions (buttons) across the bottom of the hover panel and when to display them. The display template is called Item_CommonHoverPanel_Actions and can be found here:

Site Settings > Master Pages and Page Layouts > Display Templates > Search > Item_CommonHoverPanel_Actions.html

If you inspect this display template you will find an if else block around the rendering of the POST and FOLLOW buttons. The logic can be summarised as:
The POST button is visible if Yammer is enabled and the result type supports it, otherwise the FOLLOW button is visible if the result type supports it, at no time will both buttons be visible.

If you download a copy of the display template HTML file, update it to remove the ‘else’ as in the code snippet below, and then upload it again, you will find the both the POST and FOLLOW buttons will be displayed in the search hover panel when supported. Success!

But is it okay update that file?

The short answer is yes. Take care as this file is used by every hover panel in SharePoint (to my knowledge, there may be some completely unique ones) and so changes could break something that isn’t obvious.

The major risk is that if Microsoft decide to update the hover panel which require them to produce a new version of the display template file (they have done this previously when introducing the POST button). In the case that you have modified this file, then your changes will be lost. This can happen without warning (unless you have a second tenant on first release to catch these issues before they hit production – you should be doing this!).
For very minor updates such as this, and to support non-critical functionality, it may be okay to make these changes and be prepared to re-implement them should Microsoft issue an update.

The alternative is to make a copy of the display template with a new name. This approach means that your changes will not get overridden but it also means that your solution will not get the updates that would otherwise be pushed to this file. We call this ‘customisation tax’ and it is a trade off as to which way you’d rather push changes.
In this particular scenario this latter approach is not very practical as every result type references the existing display template. You would be required to make copies of all the result type display templates that are applicable (possibly a dozen or more), and update the result types themselves to use your new templates. Unless you are bypassing result types and using a single display template for all results, this feels overly complex for such a minor change, but major changes will necessitate the effort.

EDIT: A colleague of mine, Luis Manez, pointed out that with a little JS you can force a custom hover panel to be rendered for all result types. You can read about it (approach one) and some other approaches to associating custom hover panels here (Elio Struyf).


Increase Search Result Limit Beyond 50

Content Search Web Parts (CSWPs) and Search Result Web Parts (SRWPs, a.k.a Result Script Web Parts RSWPs) prevent more than 50 results being returned at once. This is true with or without paging enabled. This is true even if you provision the web part using XML. In older versions of SharePoint, it may allow you to enter a row limit greater than 50 but will then default to a mere 10 results.


I’m sure Microsoft has very good UX and technical reasons behind this limitation, however at 100 items I don’t feel that either UX nor performance suffer. I imagine that if using this technique for returning large page sizes (>100) that there is very real potential for bad performance and the UX is most likely appalling.

Importantly – this code should only be considered a learning tool. This code is entirely unsupported and generally just a really bad idea. Please be responsible šŸ™‚

Recognition to Matt Stark who provided this solution. All I have done is rewritten it a bit for safety and I wanted to prefix it with a bit more discussion than it had. Original discussion is here.

The web parts which you want to act upon must be set to load asynchronously (this is not the default mode).
Edit Web Part > Change Query > Settings > Loading Behaviour :: Async option

Edit Web Part > Change Query > Settings > Loading Behaviour :: Async option
Edit Web Part > Change Query > Settings > Loading Behaviour :: Async option

Take note:

  1. There are a few available solutions to this, none of which are much good, and even this one is should be considered a hack and be treated with caution.
  2. I will only discuss this option as it is the best I have come across when considering the limitations of SharePoint Online.
  3. Please only include this code on specific pages using a Content Editor or a Script Editor, and NOT globally.
  4. I have amended the original code such that it only changes the row limit for those web parts which are set to return 50 items exactly.
  5. I haven’t found any issues with this code, however I am using on a page that does nothing but display a list and in a situation where failure has limited impact on the system.
  6. Apparently doesn’t work for anonymous users

And finally the code…

// <script type="text/javascript">
var CC = window.CC || {};
CC.CORE = CC.CORE || {};

CC.CORE.IncreaseSearchResultsMax = (function () {
    "use strict";

    var $ocreate = null;
    var newMaxItems = 100;
    var oldMaxItems = 50; // web part must be set to show this many items

    // on application initialization 
    // steal the global create variable and 
    // intercept calls to create UI widgets.
    Sys.Application.add_init(function() {
        $ocreate = $create;
        $create = updateResultCountCreate;

    // listen to UI widget calls for CBS & DP
    var updateResultCountCreate = function (a,b){
        var ps = Array.prototype.slice.call(arguments, 0);  
        if(a === Srch.ContentBySearch && b.numberOfItems === oldMaxItems) { 
            b.numberOfItems = newMaxItems;
        if(a === Srch.DataProvider && b.resultsPerPage === oldMaxItems) {
            b.resultsPerPage = newMaxItems;
        $ocreate.apply(this,ps);  // apply the original $create method that we stole
    return true;
// </script>


Linking to pre-filtered or ‘refined’ search results

Good news – it really is very easy to have links to pre-filtered search pages (search pages with active refiners). This post will give an example of linking to search page with a single active refiner but you should be able to extend the example to more complex scenarios if you wish.

EDIT: I have written a more recent post which provides JavaScript functions to generate these links dynamically


Cut to the chase

This is what you are looking for, I’ll explain it in depth below:

<Server Relative URL of Search Page>
<Managed Property Name>
<HEX Encoded Managed Property Value>


You can quickly perform HEX encoding using an online tool such as this.

(The above URLs have been separated on to new lines for readability but should not contain any white space)

Breaking it down

As you have probably already noticed, by applying a filter to a search results page you are appending a hash (or anchor) to the query string. This value is a URL encoded JSON object. Once decoded it looks something like this:

You can quickly performĀ URL decoding using an online tool such as this.

Importantly for our purposes we can see the name of the managed property on which we are refining “RefinableString08”.

The value isĀ a little more obfuscated. The value ‘5265706f7274’ is in fact the HEX encoded value which we want to refine on. It is vital the “t” property is set to this value prepended by the two “Ē‚” characters. The “m” property appears to define a mapping between the HEX encoded value and original value but it does not appear to be necessary.

Reducing the length to fit such that it can be used as a ‘Simple Link’ in a navigation term

A navigation term only supports a limited number of characters in its ‘simple link’ field, approximately 260. This means that the full encoded JS object as above will breach this limit if the managed property value is greater than about 8 characters (if using a ‘RefinableStringXX’ managed property). Not great. However, it turns out that the “m” property of the object is optional, at least when performingĀ simple refinement as we are here. The example I provided at the top of this post has this property excluded in order to reduce it’s length.




PowerShell Add-Type Error: Could not load file or assembly XXX or one of its dependencies.

If you are attempting to run the PowerShell command Add-Type with a Path parameter that references an assembly which has been emailed or downloaded from the internet (e.g. Add-Type -Path "C:\Microsoft.SharePoint.Client.dll") then it is likely that you will encounter the following error:

Could not load file or assembly ‘file:///’ or one of its dependencies. Operation is not supported. (Exception from HRESULT: 0x80131515)


To confirm that the following steps are appropriate. Check the inner exception:

PS> $error[0].Exception.InnerException

“An attempt was made to load an assembly from a network location which would have caused the assembly to be
sandboxed in previous versions of the .NET Framework. This release of the .NET Framework does not enable
CAS policy by default, so this load may be dangerous. If this load is not intended to sandbox the assembly,
please enable the loadFromRemoteSources switch. See http://go.microsoft.com/fwlink/?LinkId=155569 for more

Not using SharePoint Client Components?

First check the execution policy under which PowerShell is running. This can be done by running the Get-ExecutionPolicy cmdlet. If you are not running under an Unrestricted policy, try running the Add-type cmdlet after setting the execution policy to Unrestricted (Set-ExecutionPolicy “Unrestricted”). If this resolves the issue then you’ll need to look into getting the assembly signed by a trusted publisher if you can’t continue to run PowerShell in this lowered state of security.

Next ensure that the assembly hasn’t been blocked. The easiest way to achieve this is by right-clicking on the assembly and checking on the General tab that there isn’t the option to unblock the assembly at bottom of the dialog. There is also the Unblock-File cmldlet to achieve this action via PowerShell.

Unblocking an assembly
Unblocking an assembly

Finally, PowerShell supports a config file which by default is not present. It allows configuration of a few things like the .NET versions which are supported and, importantly for us, whether or not PowerShell is allowed to load from remote sources. In order to add/update this file, ensure that you are logged in as a local administrator. The config file needs to reside adjacent to the PowerShell exe file. So, by default, it ought to be created here: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe.config

Note: Do not confuse this with powershell_ise.exe.config although you may want to update this as well if you use ISE.

The content of the powershell.exe.config should be as follows. If you already have a file present, ensure that you merge the XML rather than just replacing it.

<?xml version="1.0"?>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedruntime version="v4.0.30319"/>
    <supportedruntime version="v2.0.50727"/>
    <loadfromremotesources enabled="true"/>

Using SharePoint Client Components?

If you are addingĀ the SharePoint Client assemblies required for the execution of CSOM requests then you should take a different approach. Rather than bundling these assemblies with your script, instead insure that the correct SharePoint Client Components package is installed on the executing machine and reference the assemblies from their default location:

Download Client Components:
SharePoint Online Client Components
SharePoint 2013 Client Components

Reference Accordingly:
C:\Program Files\Common Files\microsoft shared\Web Server Extensions\16\ISAPI\assembly.dll
C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\ISAPI\assembly.dll


CSS fix: SharePoint global navigation hidden by iFrame (web part)

If you have a global navigation with more than a few second level items then the fly-out nature of the menu may cause some page content to be hidden while the menu is being interacted with by users. This is expected and acceptable behaviour.

I have found that sometimes (in IE only) the navigation fly-out falls behind page elements and cannot be used correctly. This is the case when iFrames are present in the content area. This issue came to my attention when using the Yammer Embed API to provide users with Yammer feeds from SharePoint.

The global navigation fly-out is being hidden by a Yammer Embedded feed (rendered as an iFrame)
The global navigation fly-out is being hidden by a Yammer Embedded feed (rendered as an iFrame)

The simplest way to resolve this is with CSS. The following CSS snippet should resolve this issue for you. We all know about z-index but the position:relative is also required for it to be applied correctly in IE.

N.B. The below snippet is specifically for Yammer Embed iFrames, if you would like to target all iFrames then the id selector should be removed (i.e. iframe#embed-feed becomes iframe).


SharePoint Online Supporting High Resolution Background Images

When you import an image to use as a background image as part of a composed look in SharePoint Online, the image is re-scaled to 1024 x 768 and compressed as low quality jpg. If you have an image which still looks reasonable after these modifications then please continue to use composed looks as the ensures that the page weight is kept to a minimum.

A full HD image
A full HD image
The same image as above after being processed by the Composed Look framework. Admittedly the compression is pretty good in this example.
The same image as above after being processed by the Composed Look framework. Admittedly the compression is pretty good in this example.

SharePoint providesĀ no configuration around this that I am aware of and even you provide a background image URL in the composed look to a non-compressed image, the image will be compressed and the compressed image will be referenced when the composed look is applied.

The only way to support a high resolution background image is to reference it in CSS, overriding the composed look background image. I have found that the best way to do this is with the following two CSS rules. The second prevents the background being used in dialogs which occasionally don’t use the transparent overlay and can become unusable otherwise.

If the image needs to be changed later, a new image with the same name can uploaded to replace the current one. The issue with this approach is that client browsers may have cached the background image and will continue to use the cached image until the cache expires (could be years, not sure what the default cache is) or the user manually clears their browser cache. In order to force the browser to re-fetch the image a query string value should be appended (or amended, in the current example we are using ‘v=1.0’).

Note that if you are using theme-able CSS you will need to reapply the theme in order to apply changes to the CSS.


SPO CSOM Error: For security reasons DTD is prohibited in this XML document

I found myself encountering the following error when authenticating to SharePoint Online using CSOM from PowerShell:

Exception calling “ExecuteQuery” with “0” argument(s): “For security reasons DTD is prohibited in this XML document. To enable DTD processing set the DtdProcessing property on XmlReaderSettings to Parse and pass the settings into XmlReader.Create method.”

I believe that there a number of causes for this issue some of which are firewall and ISP related. This may only resolve a subset of the cases where this issue has been arising, even under the same circumstances.

In my scenario, I found that this issue was only arising when the credentials I was passing were being federated. That is, when the username was *not* in formĀ <me>@<domain>.onmicrosoft.com but rather something like <me>@<domain>.co.uk. It is also possible that this issue resolves itself after a single successful authentication has occurred. Try providing credentials for a *.onmicrosoft.com account, and if that works try again with a federated account. This is discussed more later.

I used Fiddler to compare the request/response trace from a successful authentication and one where this error occurs. It turns out that somewhere internally a request is made to msoid.<full-domain> where <full-domain> is the bit after the @ symbol from the username provided. In the case where this value is of the *.onmicrosoft.com variety, a 502 error (no DNS entry)Ā is returned with no request body and the authentication proceeds successfully. In the other case, the ‘msoid’ URL is resolved and a response with a request body is returned.
In my case the response was a 301 error (permanent relocation), however I read of cases where a 200 (success) has been received. Importantly to note, is that the response, success or otherwise, returns an HTML body containing a DTD (Document Type Declaration), and in turn produces the rather unhelpful error message.

So how do you fix it? Well one way is to provide an entry in your hosts file which ensure that the msoid URL will be invalid. I found that providing a local host entry for it worked. Your hosts file can be found here:

I added a line which looks like the following: Ā  Ā  Ā  Ā msoid.<domain>.co.uk

And it worked! Intriguingly I found that if I then removed this line from my hosts file, SharePoint Online authentication from PowerShell continued to work. It is for this reason that I suggested trying to use a *.onmicrosoft.com account first at the begging of this post – just in case itĀ resolves the issue for you without touching the hosts file. Please comment if you have any success (or otherwise) with that approach.

Hope this helps! Good luck.

SharePoint Online remote authentication (and Doc upload)

The SharePoint REST API is touted as being the tool to provide inter-platform integration with SharePoint Online. However, outside of .NET the authentication piece is not so straightforward. App authentication solves this issue for registered apps but I want to show how remote user authentication can be achieved, regardless of platform.

The goal of this post is to provide examples of the HTTP requests which need to be made in order to authenticate SharePoint Online. It then provides an example of using the same technique to upload a document and update metadata just to prove it all works šŸ™‚

The type of applications where this kind of approach may be necessary include: a Java application, a PHP application, or JavaScript application where there is otherwise no SharePoint Online authentication context and the decision has been made (for whatever reason) that user authentication is most appropriate (as opposed toĀ app authentication).

Edit: This approach will not work in a JavaScript environment due to cross-domain restrictions enforced by browsers (unless of course you are on the same domain, in which case you don’t need to worry about any of this anyway). The ADAL.js library is available for the cross-domain JS scenario. I have posted an example here: http://paulryan.com.au/2015/unified-api-adal/

I wrote about using the SharePoint REST API here (and background here, and here).

I will be providing examples of the requests using the ‘Advanced REST Client’ Google Chrome extension.


The authentication piece comes in a few steps:

  • Get the security token
  • Get the access token
  • Get the request digest

Get the security token

First we must provide a username and password of a user with Contribute access to the Roster Data library and the URL at which we want access to the SharePoint Online Security Token Service.

This is done by POSTing the following XML as the request body to:

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope"
    <a:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>
    <a:To s:mustUnderstand="1">https://login.microsoftonline.com/extSTS.srf</a:To>
    <o:Security s:mustUnderstand="1"
    <t:RequestSecurityToken xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">
      <wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">

Requesting the security token
Requesting the security token

The response from the request includes the security token needed to get the access token. It is the value which has been stricken out in orange in the image below.

Response including the security token
Response including the security token

Get the access token

Once the security token has been retrieved it must be used to fetch the access token. The can be done by POSTing to the following URL with the security token as the request body:

Request to fetch the access token, passing the security token
Request to fetch the access token, passing the security token

The response from this request includes couple of cookies which must be passed as headers with all future requests. They are marked with the ā€˜Set-Cookieā€™ header. We need the ones beginning with rtFa= and FedAuth=. They can be seen the below image of the response headers.

Response includes the access token cookies
Response includes the access token cookies

Get the request digest

The request digest is a .NET security feature that ensures any update requests are coming from a single session. It must also be included with any POST requests.

The request digest is fetched by POSTing to: https://yourdomain.sharepoint.com/_api/contextinfo
The access token cookies must be included as Cookie headers with the request as shown in the image below.

Request to fetch the request digest, passing access tokens
Request to fetch the request digest, passing access tokens

The response from the request will include the request digest in the XML response as in the image below. The entire contents of the FormDigestValue tag will required, including the date time portion and timezone offset (-0000).

Response containing the request digest value
Response containing the request digest value

Upload a document with metadata

Upload the document

Now that we have all the authentication headers we can make update calls into SharePoint Online as the user whose credentials we originally supplied when fetching the security token.

In order to upload a document perform the following POST request:
https://yourdomain.sharepoint.com/subweb/_api/web/lists/getbytitle(ā€˜list nameā€™)

A number of headers must be send with the request including the access token cookies, the request digest (X-RequestDigest) and the accept header as shown in the image below. The body of the request must contain the content of the document being uploaded.

Request to upload a document to SharePoint Online
Request to upload a document to SharePoint Online

The response of this request contains some minimal metadata about the file and can be safely ignored. However, for completeness here it is.

Response JSON from the document upload request
Response JSON from the document upload request

The unique ID property could be used to fetch the document in order to perform metadata updates rather than URL as done in the following example.

Update document metadata

The final step which needs to take place is update the document in SharePoint with the relevant metadata.

This can be done with yet another POST request. This time to the following URI:

All the headers sent with the previous request must be sent with this request as well. The request body is a JSON object which defines the metadata fields to be updated. The fieldname and fieldValue properties must be updated as required. Note that the fieldname property must be equal to the field internal name not the field display name. An example of this is in the image below.

Request to set metadata on a document in SharePoint
Request to set metadata on a document in SharePoint

The response from this request provides success notification for each individual field update as shown below.

Response from the document metadata update request
Response from the document metadata update request

So, this should now be enough to write an application in any server-side language which supports web requests and work against SharePoint Online. I’d love to see some implementations of this, please comment if you’ve done it.

I’d like to acknowledge the following posts as they were invaluable references:


SharePoint licencing limitations: Standard vs Enterpise Features and Kiosk Users

When discussing Office 365 licencing there are a number of things that as an architect or developer you must be aware of.

EDIT: A handy Excel tool for checking which features are available for different licence types can be found here: Office 365 service comparison

Image of Buying kiosk licences
Buying kiosk licences

Creating solutions for limited users
Kiosk users are the cheap users will the greatest restrictions. However the limitations placed on these users really are quite manageable in many circumstances and shouldn’t cause you particular worry when developing a solution for these users. The key points to remember when providing a solution to these users is:

  • They don’t have a user profile. They can still view the ‘My Settings’ page, but not the ‘About Me’ page. These users still have the full set of user profile properties which can be set by an administrator or via AD synch and programmed against.
  • They can only use Office Web Apps in READ mode. They cannot edit documents with a client version of the correct Office application. Kiosk users from a K2 licence (opposed to K1) can also edit documents using OWA.
  • They can’t be administrators at the tenant or site collection level. However they can be granted Full Control permissions.

Be aware of the feature set available in Production
Many features will not be present or will not work under some licencing schemes. The primary issue which I have encountered is around content rollup. Licences which do not support the Enterprise feature set do not support the Content Search Web Part. You can use the Results Script Web Part instead, but remember that the display templates used are not transferable. The Content Search Web Part display templates reference the Srch javascript namespace which will not be present if using the Results Script Web Part.
There are obviously many other Enterprise features which I won’t mention explicitly but have a browse over the below table:


Developer features Only in SharePoint Server 2013ā€”Enterprise Edition
Access Services Yes
BCS: Rich Client Integration Yes
BCS: Tenant-level external data log Yes
Custom Site Provisioning Yes
InfoPath Forms Services Yes
Analytics Platform Yes
Improved Self-Service Site Creation Yes
eDiscovery Yes
Preservation hold library Yes
Video Search Yes
WCM: Catalog Yes
WCM: Cross-site publishing Yes
WCM: Faceted navigation Yes
WCM: Image Renditions Yes
WCM: Multiple Domains Yes
WCM: Topic Pages Yes
Business Intelligence Center Yes
Calculated Measures and Members Yes
Data Connection Library Yes
Decoupled PivotTables and PivotCharts Yes
Excel Services Yes
Field list and Field Support Yes
Filter Enhancements Yes
Filter Search Yes
PerformancePoint Services Yes
PerformancePoint Services (PPS) Dashboard Migration Yes
Power View Yes
PowerPivot Yes
Quick Explore Yes
Scorecards & Dashboards Yes
Timeline Slicer Yes
Visio Services Yes
Content Search Web Part Yes
Custom entity extraction Yes
Extensible content processing Yes
Query rulesā€”advanced actions Yes
Search vertical: ā€œVideoā€ Yes
Tunable Relevancy Yes