Azure AD app wildcard Reply URL

Azure AD apps (a.k.a Azure Active Directory apps, a.k.a AAD apps) are an essential component when interacting with Office 365 data outside of SharePoint – Mail, Calendar, Groups, etc.

As an O365 developer I have found myself writing JavaScript code against AAD apps (using ADAl.js) and often, especially during development, found myself entering a long list of Reply URLs. Reply URLs must be specified for any location from which authentication to AAD occurs. From a practical standpoint this results in someone (an Azure Administrator) having to update the list of Reply URLs every time a web part is inserted into a page or a new site is provisioned which relies on an Azure AD app.

If this is not done, the user is redirected to Azure login failure with ‘The reply address … does not match the reply addresses configured for the application’.

Error when Reply URL is not correctly specified
Error when Reply URL is not correctly specified

Perhaps the following is documented elsewhere but I have not come across it – a Reply URL can be specified using wildcards!

Using a wildcard Reply URL when configuring an AAD app
Using wildcard Reply URLs when configuring an AAD app

Probably the most common use for this is to end a Reply URL with an asterisk (wildcard) which will permit any URL which begins with the characters preceding it.

e.g. https://tenant.sharepoint.com/*
This example would support any URL coming from any page in SharePoint Online from within the named tenant.

It is also possible to use the wildcard character elsewhere in the Reply URL string.
e.g. https://*.sharepoint.com/*
This example would support any URL coming from any page in SharePoint Online from within *any* tenant.

Armed with this knowledge, be responsible and limit strictly how it is utilised. The implementation of Reply URL is a security feature and it is important that only trusted locations are allowed to interact with your app. I recommend only using wildcard Reply URLs in development environments.

Paul.

Creating dynamic links to Delve pages

Delve, as part of the Office 365 suite, provides a number of useful pages for finding content or people that are trending around you or that you recently interacted with. Often, as a Developer, these pages are the perfect target for “See More” links as part of customisations written using the Office Graph. Or perhaps as an administrator you would like to configure a promoted link on a team site home page to navigate to a user’s ‘Your Recent Documents’ page in Delve, for example.

Delve recent document page
The Delve Recent Documents page. Note that the URL contains the user’s AAD object ID.

Delve Links – a minor problem

When you visit pages that show content relevant to a specific user (such as Your Recent Documents or the Recent Documents page for another user) the URL of that page contains a query string variable ‘u’ with the value of this variable equal to the Azure Active Directory (AAD) object ID of the user. Azure Active Directory is the identity provider that backs Office 365 and is out the scope of this post. If this parameter is not provided then Delve falls back to the Delve homepage. I would have preferred it to have just used the current user if the parameter is not present, but no, this is how it works.

So, you can fetch the AAD Object ID by calling into the User Profile Service (Vardhaman Deshpande), however this is not necessary.

Delve Links – an easy resolution

The ‘u’ query string parameter can be substituted for the ‘p’ query string parameter where the value of p is the user’s account name – the email address which they use to login as.

This value is present on any SharePoint 2013+ page via the JavaScript variable: _spPageContextInfo.userLoginName
This can be utilised as follows:

var mySiteHostUrl = "https://-my.sharepoint.com";
var pageKey = "liveprofilemodified"; // liveprofilemodified='Recent Documents', liveprofileworkingwith='People page'
var delveUrl = mySiteHostUrl + "/_layouts/15/me.aspx" + "?v=" + pageKey + "&p=" + _spPageContextInfo.userLoginName;

Delve Links – side note

This value is present as an Office Graph property as: AccountName
The AAD Object is present as an Office Graph property as: AadObjectId

Paul.

Convert an existing plain text note field/column to rich text

If you create a SharePoint site column (a note field in this case), associate it with a site content type, and then associate that content type with a list in a sub site, the site column will be available on that library. Obviously right?

However, when you update the site column (and push all changes to lists and libraries) not *all* of the changes you make are in fact pushed down. An example of this is the setting that dictates whether a note field should allow rich text or enforce plain text. If you change this setting at the site column level it will *not* propagate to libraries which already exist. New instances of the column (say if you associated the content type with a list for the first time) will be configured correctly, but existing list-level instances are not updated. NOTE: This is only true for properties specific to particular column type; common properties such as ‘required’ will be pushed down to existing instances of the column at the list level.

Configuring a SharePoint note field to support rich text
Configuring a SharePoint note field

So you want to change a list-level instance of a plain text note column to a rich text note column (or vice-versa, or otherwise change column specific properties or another field type)? You need to do it for every list where the column is in use. That would be very tedious to do via the SharePoint UI, but you can’t anyway. The UI only supports changing the set of common field properties (type, required, hidden, etc).

In comes PowerShell. Below you will find a script which updates a plain text note column to be a rich text note column. It is important to note that this script only updates the list-level columns and not the site column. This means that after running the script, new instances will continue to inherit the site column configuration.

The script takes advantage of recursion using delegate functions which is an approach I blogged about here: PowerShell Recursion with Delegate Functions

Credit also to Chris O’Brien’s topofscript.ps1 for the CSOM integration bit: Using CSOM in PowerShell scripts with Office 365

The script is written for SharePoint Online (and assumes that the SharePoint Online Client Components SDK is installed) but for this to work on-premises you would only need to update the referenced assemblies (v15 for 2013) and modify the code which passes the credentials.

Paul.

“SafeQueryPropertiesTemplateUrl” error when calling the SharePoint _api

If you call the SharePoint 2013 REST API in your applications ensure that any requests originating from the client are sent from the current web base URL to avoid returning a SafeQueryPropertiesTemplateUrl error.

If the current site is
https://tenant.sharepoint.com/sites/mysitecollection/subsite1/subsite2 then it is very important that you submit API requests as
https://tenant.sharepoint.com/sites/mysitecollection/subsite1/subsite2/_api
and NOT as any of:

  • https://tenant.sharepoint.com/_api or
  • https://tenant.sharepoint.com/sites/mysitecollection/_api or even
  • https://tenant.sharepoint.com/sites/mysitecollection/subsite1/_api

The reason for this is that the current user must have access to the site addressed by the base URL of the API request (the bit before the _api). If the user cannot access this site then the request will fail. Unfortunately it doesn’t fail in the manner you might expect (i.e. a 401 access denied exception). A request that fails in this manner will return a 500 error. The specific exception details are as follows:

Exception class:
Microsoft.Office.Server.Search.REST.SearchServiceException

Exception message:
The SafeQueryPropertiesTemplateUrl "The SafeQueryPropertiesTemplateUrl "{0}" is not a valid URL." is not a valid URL.

SafeQueryPropertiesTemplateUrl Error received when attempting to hit _api endpoint from web to which the current user does not have access
Error received when attempting to hit _api endpoint from web to which the current user does not have access

Paul.

HTTP 405 Error in .NET Web App

I was recently told that an web app I had developed was returning an HTTP 405 error upon being freshly deployed. It took me way too long to realise that cause of the issue came down to missing files. Specifically, the complete folder structure had been deployed however the files at the top level web root were missing. These are files are rather critical.

They are the web.config and global.asax

If you are seeing this error, ensure these files have been deployed correctly and aren’t corrupt as a first point of call.

Receiving an HTTP 405 in IE11
Receiving a 405 in IE11

For SEO HTTP 405

  • Chrome: The page you are looking for cannot be displayed because an invalid method (HTTP verb) is being used.
  • IE: HTTP 405 The website has a programming error. This error (HTTP 405 Method Not Allowed) means that Internet Explorer was able to connect to the website, but the site had a programming error.
  • Edge: HTTP 405 error That’s odd… Microsoft Edge can’t find this page

Paul.

Azure CDN integration with SharePoint, cache control headers max-age, s-maxage

After recently implementing an Azure-based solution to mitigate SharePoint Online’s poor image rendition performance by utilising Azure CDN (see Chris O’Brien’s post on this issue, see Fran R’s post on other Image Rendition issues) I’ve reached a few conclusions regarding setting appropriate cache control headers. It is important to reach a practical balance between performance and receiving updates to files.

Azure CDN logo

Before continuing it is important to understand the fundamental building blocks when using a CDN. At any time a file can be present in three location types: the blob or source file, the CDN endpoint(s), and users’ browser caches. In the case of Azure CDN, the source file must be a blob in Azure Blob Storage. Depending on the CDN/configuration it is likely that the file may be cached at many (dozens) of CDN endpoints dispersed around the globe. Without a CDN the only consideration is the cache timeout for files stored at the user’s browser cache. When considering a CDN we must also consider the cache timeout between the CDN endpoint and the source file.

Another important point to call out is that CDNs generally only push content to an endpoint when is it first requested: on-demand. This will incur a delay for the first user to request that asset from a given endpoint, while source blob is transferred to the endpoint. The impact of this will differ depending on the distance between the source blob and the CDN endpoint and the file size. It is this process that increasing the s-maxage header prevents (discussed below).

Relevant cache control headers

Definitions

  • max-age : Defines the period which, until reached, the client will used the cached file without contacting the server. ‘Client’ refers to a user’s browser cache as well as a CDN.
  • s-maxage : If provided, overrides max-age for CDNs only
  • public : Explicitly marks the file as not user specific
  • no-transform : Proxy servers may compress or encode images to improve performance or reduce bandwidth traffic. This header prevents this for occurring. It is preferable to avoid this header assuming that you can spare the effort to ensure the files being served are not affected adversely.

A good summary of the many remaining cache control headers that I didn’t feel were relevant to this post can be found here:
A beginners guide to HTTP cache headers

In practice

  • For an image that has been previously requested:
    • When s-maxage has not expired and max-age has not expired, server responds with 200 (OK), the file is not downloaded again [0ms]
    • When s-maxage has not expired but max-age has expired, server responds with 304 (not modified), the file is not downloaded again [<100ms]
    • When s-maxage has expired but max-age has not expired, server responds with 200 (OK), the file is not downloaded again [0ms]
    • When s-maxage has expired and max-age has expired and the blob has not changed, server responds with 304 (not modified), the file is not downloaded again [<100ms]
    • When s-maxage has expired and max-age has expired and the blob has changed, server responds with 200 (OK), the file is downloaded again [download image]
  • A request for an image will return 200 (OK) until max-age has expired and then 304 (not modified) for every subsequent request until the blob is updated. Once updated, this process repeats
  • If an existing image is updated, the longest a user can wait to see the updated image is
    • Without clearing browser cache: max-age + s-maxage
    • With clearing browser cache: s-maxage
  • If an user views an image from the CDN for the first time, it is only guaranteed to be the latest version of that image if the blob hasn’t been updated in the last s-maxage
  • SharePoint library images are served with a max-age of 24 hours
  • As SharePoint library images are not served via a CDN they have an effective s-maxage of 0

My recommendations

Keeping all of the above in mind, I feel that the most important factor is to replicate the experience that users expect from images being served from the SharePoint environment. This can presented as a couple of simple rules:

  1. max-age + s-maxage = 24 hours = 86400 seconds
  2. s-maxage is as low as possible whilst satisfying bandwidth and performance targets (especially for locations most distant to the source blob)

For a recent SharePoint/CDN, I used the following cache control headers:

  • max-age: 23 hours
  • s-maxage: 1 hour
  • public
  • no-transform

Which looks like this:
no-transform,public,max-age=82800,s-maxage=3600

Setting the cache headers served by Azure CDN and Azure Blob Storage

When working with cache control headers in Azure, they are set on the blob itself. It is not a CDN configuration setting.

Paul.

Azure Service Management REST API vs Azure API Management REST API

Do not be confused! The Azure Service Management REST API and the Azure API Management REST API are completely different. Yes, they may have confusingly similar names but they service completely different purposes, support different authentication protocols, and are surfaced via different endpoint domains.

Question mark

The Azure Service Management REST API

What can I do with it?

This service supports actions for managing Azure resources such as web apps or storage accounts. Think of it as an endpoint for the actions you might otherwise perform manually via the (Classic or New) Azure Portal.

What do the endpoints look like?

Service request URIs will be of the form: https://management.azure.com /subscriptions/…

How does authentication work?

Service authentication is achieved using OAuth via the use of a Bearer access token in the Authorization header. The app principal is an Azure Active Directory application. The AAD app must be given ‘permissions to other applications’ for ‘Windows Azure Service Management API’. As the only grant-able permissions are ‘delegated permissions’ (App+User) rather than ‘application permissions’ (App-only), this API can only be called from within a user context and not, for example, from the context of a web job.

Configuring Azure AD App Permissions
Configuring AAD App Permissions

The Azure API Management REST API

What can I do with it?

The API Management Service supports publishing APIs to consumers by providing an ID and secret key ‘shared signature’ authentication mechanism very similar to that used by Amazon or Instagram for their (public, pending approval) APIs. An API Management Service instance provides benefits like management of users, groups, products (endpoints), and subscriptions. There is then a REST API for managing these users, groups, products, and subscription that the API Management Service provides – this is referred to as the API Management REST API.

What do the endpoints look like?

Service request URIs will be of the form: https://{servicename}.management .azure-api.net/…

How does authentication work?

Service authentication is achieved via the use of a Shared Access Signature access token in the Authorization header. The identifier and secret key required to generated a request signature are available via API Management Service instance. Access to the API must be explicity allowed by checking the ‘Enable API Management REST API’ via the API Management Service publisher portal.

Enable API Management REST API
Enable API Management REST API
Credit to Microsoft Azure Documenation

Read more

Ok, so just reading the above really won’t be enough to get you firing off requests but hopefully it will provide enough clarity that you fully understand how to interact with which API.

Service Management REST API Reference

API Management REST API

Vaguely related, I’ve recently blogged about posting authenticated requests to the Amazon Product Advertising API in .NET

I will post about using the Service Management API along will app configuration and full code samples for authentication in the near future. I will link to that post from here.

Paul.

PowerShell recursion with delegate functions, iterate all lists in all webs

There are many ways to iterate a collection in PowerShell. I just really like using delegate functions. This approach is not native PowerShell but utilises the .NET Action class as a function parameter. Using a delegate function approach, it is possible to create a recursive loop that can be very easily reused in the future just by providing an alternative Action.

The example code I provide below demonstrates how to create a delegate function in PowerShell, how to write a function that accepts one as a parameter, and provides some ready made samples for iterating SharePoint objects, specifically all webs or all lists. I am using some specific SharePoint objects in these samples, however the fundamental pattern can be used to effectively iterate any recursive structure.

foreachDecendentWeb : perform an action on every web below the provided web
foreachListInWeb : perform an action on every list in the provided web
foreachListInWebAndAllDecendentWebs : perform an action on every list in the current and all decendent webs

Delegates

Some notes

The below script references ‘TopOfScript.ps1’, it is specifically related to calling SharePoint CSOM from PowerShell. Read about it here on sharepointnutsandbolts.

Making the call, providing the delegate

The utility scipts, recursive functions accepting delegate parameters

Paul.

Request Signing, Amazon Product Advertising API, .NET C#

The Amazon Product Advertising API documentation provides some code samples for its use but none using ASP.NET. A personal interest brought me to play with it and as it wasn’t entirely trivial to create a signed request as required associate authentication I thought I’d share some working code samples.

Amazon Product Advertising API

Some notes

The API does surface a WSDL file and as such a Web Reference could be used to generate classes to interact with the API. The sample I am providing here does not take advantage of this and is instead submitting raw REST requests.

I see the most valuable part of this sample as the request signing piece. This sample should not be seen as a best practice for interacting with the API but rather as a utility for request signing.

The order of the query string parameters that are included in the signed string is crucial. They must be ordered by character code (in practise this equates to alphabetically, but with all upper case letters coming before any lower case letters). The API documentation suggests string splitting, sorting, and string joining. This is definitely the approach I would take if you find yourself writing queries that use parameters dynamically but I struggle to see the use-case. This sample just uses a hard-coded string with the relevant parameters in the correct order.

Although I haven’t looked in detail yet, the approach taken to sign requests here appears very similar if not identical to that required by the Instagram API, and I am sure many other (social media) APIs.

Requests to APIs which require the signing of a secret key cannot be made securely directly from the client (e.g. using JavaScript) as it would require your secret key to be available in plain text on the client. If you want to run ajax commands against the API you need execute requests to an intermediary service. This is the approach that the sample code below facilitates.

You can read about the Amazon Product Advertising API here: Product Advertising API

The code

Below you will find a class called the AmazonApiHelper. Further below is an ashx HttpHandler as an example of calling the utility functions provided by the helper class. You’ll need to provide you own values for the following constants:
private const string awsSecretKey = "Your secret key goes here";
private const string awsAccessKeyId = "Your access key Id goes here";
private const string associateTag = "Your associate tag goes here";

The helper class

 

Calling the helper class from a web handler

Good luck advertising those products!

Paul.

jsLink: how to display a custom ‘no items’ message

If you use jsLink to override the rendering of list views then you may have noticed that your custom jsLink no longer renders a message when there are no items returned in the view. I am going to discuss with code samples how to display a ‘no items’ message – or at least help you stop overriding it.

You have complete control over how list items are rendered using jsLink
You have complete control over how list items are rendered using jsLink

If, alternatively, you have a ‘no items’ message being displayed and just want to modify the text, try this link.

If you don’t know what jsLink is then it is worth learning about it. Try this link.

What am I doing wrong?

Chances you are making the same mistake that many people make. A mistake that has been replicated again and again online and doesn’t break anything but does prevent the display of the ‘no items’ message and the paging control. When you override Templates.Header you DO NOT need to override Templates.Footer in order to close tags which you opened in the header.

Although doing so seems to make sense, you can rest assured knowing that tags you open in the header will be closed auto-magically after the item templates have completed rendering. In fact, the footer template is rendered in a different table cell to the header and item templates when this all hits the page. Think of the footer template as a distinct block that is rendered after everything else rather than the end of the same block.

By overriding the footer template you are also inadvertently overriding the ‘no items’ message and the list view paging control. You can see exactly what you are overriding by inspected the default values for the templates. Below is snippet from clientrenderer.js which shows the default footer template.

So what should you do?

If you just want the default no items message and can get away with not overriding the footer template (as in the first code snippet), then great – you are all done.

If want a custom message then check out the link at the very top of the article (in summary: renderCtx.ListSchema.NoListItem = "Nada, nothing, zilch";).

If you want to override the footer template or perhaps you want the message to appear within a wrapper tag defined in the header or you want some custom logic behind which message to display then you can do that too – keep reading.

Doing it yourself

I’ve written a utility function that is based on the logic in the OOTB footer template that makes it easier to manage the ‘no items’ text. This function does NOT replicate the paging functionality. If you need paging and are overriding the footer template then you will need to replicate the paging functionality as well. You will need to look into clientrenderer.js to find out how MSFT do this.
Looking at this snippet you can see the if-else block where you can define custom messages for different list templates or if the lack of results has occurred only after a search term was provided. This sample should not be considered the superlative version, it just does a basic job in line with what happens by default.

Below are two examples of how you may want to use this. The first is by overriding the footer template, and the second is by overriding the header template. The advantage of sticking this code into the header template is that it allows you wrap the no items message in the same wrapper tags that you defined for the main content.

Paul.

For aiding findability:

  • There are no items to show in this view of the list
  • Your search returned no results
  • Some items might be hidden. Include these in your search
  • Still didn’t find it? Try searching the entire site.

SharePoint – discoveries while "working on it …"