If you call the SharePoint 2013 REST API in your applications ensure that any requests originating from the client are sent from the current web base URL to avoid returning a SafeQueryPropertiesTemplateUrl error.
If the current site is https://tenant.sharepoint.com/sites/mysitecollection/subsite1/subsite2 then it is very important that you submit API requests as https://tenant.sharepoint.com/sites/mysitecollection/subsite1/subsite2/_api
and NOT as any of:
https://tenant.sharepoint.com/_api or
https://tenant.sharepoint.com/sites/mysitecollection/_api or even
The reason for this is that the current user must have access to the site addressed by the base URL of the API request (the bit before the _api). If the user cannot access this site then the request will fail. Unfortunately it doesn’t fail in the manner you might expect (i.e. a 401 access denied exception). A request that fails in this manner will return a 500 error. The specific exception details are as follows:
Do not be confused! The Azure Service Management REST API and the Azure API Management REST API are completely different. Yes, they may have confusingly similar names but they service completely different purposes, support different authentication protocols, and are surfaced via different endpoint domains.
The Azure Service Management REST API
What can I do with it?
This service supports actions for managing Azure resources such as web apps or storage accounts. Think of it as an endpoint for the actions you might otherwise perform manually via the (Classic or New) Azure Portal.
What do the endpoints look like?
Service request URIs will be of the form: https://management.azure.com /subscriptions/…
How does authentication work?
Service authentication is achieved using OAuth via the use of a Bearer access token in the Authorization header. The app principal is an Azure Active Directory application. The AAD app must be given ‘permissions to other applications’ for ‘Windows Azure Service Management API’. As the only grant-able permissions are ‘delegated permissions’ (App+User) rather than ‘application permissions’ (App-only), this API can only be called from within a user context and not, for example, from the context of a web job.
Configuring AAD App Permissions
The Azure API Management REST API
What can I do with it?
The API Management Service supports publishing APIs to consumers by providing an ID and secret key ‘shared signature’ authentication mechanism very similar to that used by Amazon or Instagram for their (public, pending approval) APIs. An API Management Service instance provides benefits like management of users, groups, products (endpoints), and subscriptions. There is then a REST API for managing these users, groups, products, and subscription that the API Management Service provides – this is referred to as the API Management REST API.
What do the endpoints look like?
Service request URIs will be of the form: https://{servicename}.management .azure-api.net/…
How does authentication work?
Service authentication is achieved via the use of a Shared Access Signature access token in the Authorization header. The identifier and secret key required to generated a request signature are available via API Management Service instance. Access to the API must be explicity allowed by checking the ‘Enable API Management REST API’ via the API Management Service publisher portal.
Enable API Management REST API Credit to Microsoft Azure Documenation
Read more
Ok, so just reading the above really won’t be enough to get you firing off requests but hopefully it will provide enough clarity that you fully understand how to interact with which API.
I will post about using the Service Management API along will app configuration and full code samples for authentication in the near future. I will link to that post from here.
Yammer and SharePoint Online are becoming more and more integrated. Recently, with the Yammer Embed widget supporting SSO from Office 365 to Yammer, we are in a situation where we can perform actions against Yammer from SharePoint Online without requiring further authentication.
This opens up opportunities for utilising the Yammer SDK and Yammer REST API to build all kinds of Yammer interactions directly into your SharePoint pages. It also allows us to start implementing some of those anti-patterns that customers want but Yammer doesn’t want to support as they’re against ‘social freedom’. A prime example of this is forcing users into groups. In some scenarios it may be rather practical. I won’t discuss the pros and cons of this further but do consider that Microsoft would rather you coerce users socially to make their own decision to join the ‘correct’ groups rather than programmatically deciding for them.
Regardless of that, I’m going to provide an example which, upon page load, joins the current O365 user’s Yammer identity to a Yammer group based upon their SharePoint user profile.I’d like to point out that if Yammer Embed is present on the page and is enabled with Single Sign On, then the authentication piece can be hidden entirely from the user. I am currently unaware how to achieve SSO with Yammer from SharePoint Online without pigging-backing Yammer Embed, although I haven’t looked in earnest so I suspect it’s achievable without too much effort.
A few notes on the code:
Add the first code snippet to a page with a Script Editor web part. It calls the initiation code and any configurations can be provided here, and modified when live
The second code snippet contains all the logic. This can be included to the page in any manner you wish but you must ensure it has loaded prior to running the init function
In order to use the Yammer SDK you must register a Yammer app on the target network and provide the client ID as the data-app-id attribute on the script element which includes it
Each user must authorise the Yammer app, just once, before it can act in their behalf. I have implemented this as a status message, an example of which can be seen in the image below
Each user must authorise the Yammer App to act on their behalf
The code references ‘hut Id’ which is just a value stored in user’s profile and which is used to map a user to a Yammer group
I use local storage to prevent the code executing more often than every 24 hours. This has been commented out for clarity, however I would recommend functionality such as this is re-implemented
The experience of signing into Yammer from SharePoint is different if SharePoint is hosted on-premise or online. Only when online is the same identity used and can an SSO experience be achieved. In contrast, on-prem, the disconnect between O365 and Yammer credentials allows users to to provide credentials for any Yammer user in any Yammer network rather than being restricted to the associated identity
And finally, the code:
Finally, for completeness, here is a the settings object which I pass to Yammer Embed to achieve SSO with Yammer from SharePoint Online. I find that in practice anywhere I would want to run the above code I also have a feed of some sort that is appropriate to display. If this is not the case for you, hiding the feed with display:none will achieve the same result as long as the width of the Yammer Embed is equal to or greater than 400px. Note that this is *not* required, however without it the user may be prompted to provide their Yammer credentials.
The SharePoint REST API is touted as being the tool to provide inter-platform integration with SharePoint Online. However, outside of .NET the authentication piece is not so straightforward. App authentication solves this issue for registered apps but I want to show how remote user authentication can be achieved, regardless of platform.
In a .NET environment please refer to the ADAL library for authentication rather than writing it yourself.
The goal of this post is to provide examples of the HTTP requests which need to be made in order to authenticate SharePoint Online. It then provides an example of using the same technique to upload a document and update metadata just to prove it all works 🙂
The type of applications where this kind of approach may be necessary include: a Java application, a PHP application, or JavaScript application where there is otherwise no SharePoint Online authentication context and the decision has been made (for whatever reason) that user authentication is most appropriate (as opposed to app authentication).
Edit: This approach will not work in a JavaScript environment due to cross-domain restrictions enforced by browsers (unless of course you are on the same domain, in which case you don’t need to worry about any of this anyway). The ADAL.js library is available for the cross-domain JS scenario. I have posted an example here: https://paulryan.com.au/2015/unified-api-adal/
I wrote about using the SharePoint REST API here (and background here, and here).
First we must provide a username and password of a user with Contribute access to the Roster Data library and the URL at which we want access to the SharePoint Online Security Token Service.
This is done by POSTing the following XML as the request body to: https://login.microsoftonline.com/extSTS.srf
Requesting the security token
The response from the request includes the security token needed to get the access token. It is the value which has been stricken out in orange in the image below.
Response including the security token
Get the access token
Once the security token has been retrieved it must be used to fetch the access token. The can be done by POSTing to the following URL with the security token as the request body: https://yourdomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0
Request to fetch the access token, passing the security token
The response from this request includes couple of cookies which must be passed as headers with all future requests. They are marked with the ‘Set-Cookie’ header. We need the ones beginning with rtFa= and FedAuth=. They can be seen the below image of the response headers.
Response includes the access token cookies
Get the request digest
The request digest is a .NET security feature that ensures any update requests are coming from a single session. It must also be included with any POST requests.
The request digest is fetched by POSTing to: https://yourdomain.sharepoint.com/_api/contextinfo
The access token cookies must be included as Cookie headers with the request as shown in the image below.
Request to fetch the request digest, passing access tokens
The response from the request will include the request digest in the XML response as in the image below. The entire contents of the FormDigestValue tag will required, including the date time portion and timezone offset (-0000).
Response containing the request digest value
Upload a document with metadata
Upload the document
Now that we have all the authentication headers we can make update calls into SharePoint Online as the user whose credentials we originally supplied when fetching the security token.
In order to upload a document perform the following POST request: https://yourdomain.sharepoint.com/subweb/_api/web/lists/getbytitle(‘list name’)
/rootfolder/files/add(url='filename.csv',overwrite=true)
A number of headers must be send with the request including the access token cookies, the request digest (X-RequestDigest) and the accept header as shown in the image below. The body of the request must contain the content of the document being uploaded.
Request to upload a document to SharePoint Online
The response of this request contains some minimal metadata about the file and can be safely ignored. However, for completeness here it is.
Response JSON from the document upload request
The unique ID property could be used to fetch the document in order to perform metadata updates rather than URL as done in the following example.
Update document metadata
The final step which needs to take place is update the document in SharePoint with the relevant metadata.
This can be done with yet another POST request. This time to the following URI: https://yourdomain.sharepoint.com/subweb/_api/web/lists/getbytitle(‘listTitle')
/rootfolder/files/getbyurl(url='serverRelFileUrl')/listitemallfields/validateupdatelistitem
All the headers sent with the previous request must be sent with this request as well. The request body is a JSON object which defines the metadata fields to be updated. The fieldname and fieldValue properties must be updated as required. Note that the fieldname property must be equal to the field internal name not the field display name. An example of this is in the image below.
Request to set metadata on a document in SharePoint
The response from this request provides success notification for each individual field update as shown below.
Response from the document metadata update request
So, this should now be enough to write an application in any server-side language which supports web requests and work against SharePoint Online. I’d love to see some implementations of this, please comment if you’ve done it.
I’d like to acknowledge the following posts as they were invaluable references:
“The Open Data Protocol (OData) is a Web protocol for querying and updating data”
as defined on the OData site here. From a web developer’s point of view, I think it can be better defined as such:
‘A standardized syntax for RESTful web services’
As such, if you don’t have solid understanding of RESTful services and calling RESTful services with jQuery then I suggest you read the previous post in this series which addresses that topic specifically. I have also written a blog on OData in SharePoint 2013 specifically which assumes the knowledge laid out here.
OData protocol – agnostic of implementation
OData logo
OData defines a standard syntax which defines how resource representations (entities) are queried via a RESTful URI: [the base URI (eg http://server/service.svc)] + [the resource path (eg /departments)] + [the query options(eg ?$select=name)]. A standardised approach makes the service easier to work with for anyone with previous OData experience. The resource path can be determined as it is structured logically and the query options, once learned, are repeatable across services.
The resource path is used to locate a collection of entities, a single entity or a single property that the service surfaces in a hierarchical manner. E.g.
http://server/service.svc/departments – collection of departments
http://server/service.svc/departments(1) – single department
http://server/service.svc/organisation – single organisation
http://server/service.svc/organisation/name – name of the organisation
It’s worth keeping in mind that the OData protocol is fundamentally similar to SQL as it aims to achieve the same goal of data access. It is the query options that allow us to perform a subset of SQL-like queries: select ($select), where ($filter), order by ($orderby), join ($expand) and paging ($top and $skip) operations. E.g.
http://server/service.svc/departments?$select=name – returns only the name element for each department
http://server/service.svc/departments?$filter=name eq HR – returns only the HR department element
http://server/service.svc/departments?$orderby=name – returns all department elements sorted by name
http://server/service.svc/departments?$expand=directors – returns the department elements and including the department director
http://server/service.svc/departments?$skip=40&top=20 – returns the third page of 20 results
Multiple query options can be used at once (?$filter=staffCount gt 100&$select=name) to create complex queries. Note that a dollar sign ‘$’ character is prepended to each of OData query option parameters.
OData compliant service being consumed directly in IE
The filter query option may also support a range of math, string and date operations. It is important to remember, however, that an OData service has no obligation to support all (or any) query options.
Using the OData $filter query parameter – Thanks to Ted Pattison’s SPC12 slide deck for this imageUsing the OData $filter query parameter – Thanks to Ted Pattison’s SPC12 slide deck for this imageUsing the OData $filter query parameter – Thanks to Ted Pattison’s SPC12 slide deck for this image
Fundamentally, that is OData. The specification has a lot more to it but if you are comfortable with it at this level then it is just a matter or reading up on the spec. The blog post here expands a little on the fundamentals I’ve just described or the full spec can found here but it’s a rather difficult read as it’s raw unstyled HTML.
The goal of this article is to provide a practical understanding of RESTful services with jQuery as it is a fundamental prerequisite to working with the OOTB OData implementation in SharePoint 2013. This is the first part of a series of posts on OData in SharePoint 2013. The other parts are Understanding OData and OData in SharePoint 2013 with jQuery.
For those of you who aren’t familiar with RESTful services, it is important that you first have an understanding of this technology as it’s the basis of the OData protocol. It’s also important to have experience using jQuery to interface with services now that Microsoft advocates this approach. If you already have these skills then you may want to skip ahead to the next post in this series.
jQuery logo – sourced from www.jQuery.com
Understanding RESTful services in jQuery
REST (REpresentational State Transfer) is a design model (or architectural style) for web services. Services implementing the REST design model are considered as being ‘RESTful’. These services are accessed via HTTP URIs (as opposed to complex mechanisms like SOAP) and provide a stateless, human readable and structured way of accessing data. REST services can be utilized from a web client with only fundamental knowledge of JavaScript and JSON (or XML) and as such are inherently cache-able These services can support CRUD (Create Read Update Delete) by leveraging well known HTTP methods – GET, POST, PUT and DELETE – and hence limit the chance of being interferred with by a firewall. So that’s all lovely, but what does it look like?
That was an example of a call to a RESTful service. We will go over it piece by piece but before that I should start by making it abundantly clear that this is jQuery – you must have included a script reference to the jQuery library. Of course you could go back to basics and implement a solution in raw JavaScript (or many other client-side technologies) if you have a specific requirement. I would think very hard before making that decision. JQuery has been adopted by Microsoft and is now included as part of many Visual Studio item templates (including SP Apps and Office 2013 Apps) and greatly improves developer efficiency as well as largely mitigating browser compatibility issues.
The $.ajax is the magic method, but before we call it we have to define an object which represents the settings as to how the ajax call should be performed.
The url property
Of primary importance is the url property. This must identify an endpoint capable of handling the HTTP request. The endpoint in this example is an HTTP handler (.ashx) but it could be implemented in one of a number of ways including as a .NET web service (.asmx), a WCF web service (.svc) or any of a number of ways outside of the Microsoft stack. Implementing services is beyond the scope of this post, however HTTP handlers are great place to start for simple services being consumed from within your web applications but you’ll want to look into WCF services for large scale and public facing API services. In theory the endpoint could even be an aspx file but it would be impractical and has a high overhead.
A REST URI is made up of three parts: [the base URI (the end point file as just discussed)] + [the resource path] + [the query options].
The resource path in this example is ‘/items’ and defines the resource. You’ll see more on this when we get to OData. Lastly, the query string allows us to refine the result set either specifically as in this case (get me the resource with an ID equal to 8) or notionally (e.g. get me resources that have been modified in the last 12 hours).
The type property
The type property defines the action to be performed as outlined next:
GET allows us to Read resource representations
POST allows us to Create new resources
PUT allows us to Update existing resources
DELETE allows us to Delete existing resources
In the case of GET and DELETE the URI itself provides enough information to perform the action. For POST and PUT a representation of the resource is required to be sent through the wire. This is where the data property comes into play.
The data property
The data property is quite self-explanatory and as previously mentioned should only be included when performing a create or update activity. Data can be sent in a variety of formats (see below) however I would recommend using a JSON representation as it is extremely lightweight, human-readable, easily created, easily serialised (stringified) and sent, and easily deserialised (parsed) and worked with. JQuery also makes working with XML reasonably straightforward should you decide to go in that direction.
Visual representation of a JSON object – sourced from www.json.org
The contentType property
Paired with the data property is the contentType property. It is only required when then data property is present and it tells the service what kind of data to expect. Some services may work without this (especially if they only accept one data type) but it should always be included. Most commonly one would expect this value to represent json or xml but a service could be written to accept pretty much anything. However, accepting the likes of SOAP would erode the goal of a RESTful implementation. The value for this property should be an Internet Media Type. For a full list of all available types see here but it is most likely that the only ones you’ll need are these:
application/json
application/xml
text/html
The accept header property
The properties discussed so far are all defining the request. The remaining properties define the response (or at least how to handle the response). Defining the accept header lets the service know what kind of data you expect it to return. It should be noted that the service must take the responsibility of reading the accept header and generating the appropriate output. There is no inherent transformation occurring based on this value and services may only support limited response types.
The dataType property
The dataType property defines the type of data that is returned. This value will determine the transformation that is applied to the data returned from the service. If the service returns a stringified JSON object then a dataType value of ‘json’ will mean that the success callback method will receive a JSON object as the returned value. A value of ‘xml’ will return an xml document that can then be parsed easily using jQuery. Other values include ‘text’, ‘html’ and ‘script’ and can be further investigated via the jQuery API documentation.
The callback properties
Finally there are the two callback functions: one for success and one for when an error occurs. Without the success callback a GET call is pointless as it is here that you would define the action to take with the service data (formatted according to dataType). You may decide to omit the success callback for a POST, PUT or DELETE if you don’t need user feedback (but this is very unlikely). Hopefully you understand the premise of callbacks because I’m not going to explain these further (if not, you’ll have no trouble finding good references elsewhere on the web). If you want to find out more about the $.ajax function in general (not specific to REST) try visiting the jQuery API documentation here.
A couple of tips:
When testing with any web services using jQuery, make sure you run $.ajax({cache: false}); first to prevent caching responses.
You can type GET requests straight into your browser to view the xml/json response for ease of testing. However, you need to prevent the browser from treating it as an RSS feed. In Chrome prepend ‘view-source:’ to the URI (eg view-source:http://hostname) and in IE go: Tools > Internet Options > Content > Feeds and Web Slices > Settings > uncheck ‘Turn on feed reading view’
