OData in SharePoint 2013 with jQuery

This is the third part of a series of posts and assumes knowledge of RESTful Services in jQuery and Understanding OData.
In the previous post I showed an example of calling an OData service in the browser using the URI: http://server/_api/web/lists?$filter=startsWith(Title, 'Calendar')&$select=Hidden. This is an example of calling the SharePoint 2013 OData web service or as it commonly known: the ‘_api’ (underscore A.P.I). The _api is, in fact, just an alias (or mapping) to a service that was already present in SharePoint 2010: _vti_bin/client.svc. Calling this service directly wasn’t supported in SharePoint 2010 (it wasn’t OData or REST compliant until now) but if you have had any experience coding against the Client Side Object Model (CSOM) then you have been calling this service indirectly. It is important that you always use the _api alias as Microsoft provides no guarantee that the service end point won’t be moved or renamed in future releases. Side note: The SP2010 asmx web services are deprecated in SP2013 and although ListData.svc is not officially deprecated the new _api should be used going forward.

SP OData service diagram
SP OData service diagram – Thanks to Ted Pattison’s SPC12 slide deck for this image

Thankfully Microsoft decided that OData support would be useful and I think that the vast majority of developers will now rarely use the CSOM when writing SharePoint Apps or other SharePoint interfacing applications (at least from web clients; it may still have a place in desktop applications). The service API has been extended to cover:

  • Search
  • User Profiles
  • BCS
  • Taxonomy
  • Workflow
  • Publishing
  • More… (including analytics, feeds, IRM and e-discovery)

All the new functionality that is available via the RESTful interface is all present via the CSOM as well, however, the inter-platform and familiar nature of a RESTful service greatly reduces the learning curve for developers compared to the CSOM.

So, now that you have an understanding of REST and OData, the challenge of efficiently leveraging the _api service comes down to learning the service ‘syntax’. That is; learning the resource paths and query options that will return the data (SharePoint assets) which you require. To begin with you need to select an access point dependent on what you are looking for.

Service Access Points

  • Site http://server/site/_api/site
  • Web http://server/site/_api/web
  • User Profile http:// server/site/_api/SP.UserProfiles.PeopleManager
  • Search http:// server/site/_api/search
  • Publishing http:// server/site/_api/publishing

From an access point you can traverse objects using the same hierarchy which you will be familiar with having coded against the server-side API. For example /_api/web/lists/GetByTitle('Calendar')/fields/getbyid('<guid>') would return information about a single field on the Calendar list. The ‘methods’ which the SharePoint OData service use (eg GetByTitle and GetById) are extensions to the OData protocol to make the service more usable. If you have experience with the CSOM you can leverage this knowledge as the OData service can be called using syntax that is very similar. A good reference on this can be found here.

Rather that going into great depth regarding the _api vocabulary I will present practical code examples to perform GET, ADD, DELETE and UPDATE actions. Once you have these examples up and running you can look elsewhere for the syntax to find specific objects if it hasn’t been covered here.

Debugging Tip
Before you start debugging these examples keep in mind that if you press F5 to test your project it will attach the Visual Studio debugger and prevent the browser debugger from being allowed to attach. Instead press CNTL+F5.

GET Example

function onGetCustomer(customerId) {
 
  // begin work to call across network
  var requestUri = _spPageContextInfo.webAbsoluteUrl +
                "/_api/Web/Lists/getByTitle('Customers')/items(" + customerId + ")";
 
  // execute AJAX request
  $.ajax({
    url: requestUri,
    type: "GET",
    headers: { "ACCEPT": "application/json;odata=verbose" },
    success: function(data){
      alert(data.d.FirstName);
    },
    error:  function(){
      alert("Failed to get customer");
    }
  });
}

The _spPageContextInfo JavaScript object has been around since SharePoint 2007. It provides really easy way to start building the correct _api URI, as shown. It also includes a property for the current user Id among other values. The ACCEPT header is required when performing service call against the _api. I’ve explained the purpose of the ACCEPT header in a previous post but the point of note here is that the odata=verbose appended to the end is required in order for the service call to succeed. This is because the OData specification also supports odata=light (the default) but the _api doesn’t (as yet).

ADD Example

function onAddCustomer() {

    var requestUri = _spPageContextInfo.webAbsoluteUrl +
              "/_api/Web/Lists/getByTitle('Customers')/items/";

    var requestHeaders = {
      "ACCEPT": "application/json;odata=verbose",
      "X-RequestDigest": $("#__REQUESTDIGEST").val(),
    }

    var customerData = {
      __metadata: { "type": "SP.Data.CustomersListItem" },
      FirstName: "Paul"
    };

    requestBody = JSON.stringify(customerData);

    $.ajax({
      url: requestUri,
      type: "Post",
      contentType: "application/json;odata=verbose",
      headers: requestHeaders,
      data: requestBody,
      success: function(){
        alert("Created customer");
      },
      error: function(){
        alert("Failed to create customer");
      }
    });
  }
}

Explaining the form digest beyond “a security feature of ASP.NET” is outside of the scope of this article. The important thing to understand is that you must send the form digest along with any POST request or nothing will get updated. Luckily for us, the form digest is stored as the value of a well known element on the page: the element with id ‘__REQUESTDIGEST’ (the double underscore is not a typo). It must be passed as the value of the X-RequestDigest header, as shown.
This post is specifically about calling the _api with jQuery, however I will mention that in order to get the form digest in another context you would need to first perform a call to _api/contextinfo where it can be retrieved.

When adding a new entity you must provide the __metadata 'type' property with the list item entity type of the list. You can think of this like the content type of the list. The name of the list item entity type can often be predicted as it’s built based on the list name. E.g If you have a list called Customers then it will have list item entity type of “SP.Data.CustomersListItem”. However, this quickly ceases to hold true; If a second Customers list were to be created then the list item entity type for it would be different (an integer gets inserted). As such, it is bad practice to hard code this value and instead it should be the result of previous service call. However for the sake of code brevity I haven’t heeded my own warning in this example. In order to obtain the list item entity type you will need to perform a GET query using the resource path to the list with a URI like /_api/Web/Lists/getByTitle('Customers') and then inspect the ListItemEntityTypeFullName property of the resulting data object.

DELETE Example

function onDeleteCustomer(customerId) {

  var requestUri = _spPageContextInfo.webAbsoluteUrl +
                "/_api/Web/Lists/getByTitle('Customers')/items(" + customerId + ")";

  var requestHeaders = {
    "ACCEPT": "application/json;odata=verbose",
    "X-RequestDigest": $("#__REQUESTDIGEST").val(),
    "If-Match": "*" // delete the item even if another user has updated it since we last fetched it
  }

  $.ajax({
    url: requestUri,
    type: "DELETE",
    contentType: "application/json;odata=verbose",
    headers: requestHeaders,
    success: function(){
      alert("Deleted customer");
    },
    error: function(){
      alert("Failed to delete customer");
    }
  });

}

SharePoint supports an optimistic concurrency scheme which is what prevents users overwriting one-another’s changes when working concurrently. It works by only updating items which at the same version as when you last fetched it. This means that if another user updates the item before you can submit your changes, then your update will fail rather than wiping the other user’s changes. The If-Match header must define the eTag (version) which can be updated by the call. The “*” allows us to ignore the scheme and just overwrite no matter what. This is okay in this scenario only because we are performing a delete action. When performing an update you will need to query for the latest eTag first.

UPDATE Example

function onUpdateCustomer(customerId, firstName) {
  // first we need to fetch the eTag to ensure we have the most recent value for it
  var requestUri = _spPageContextInfo.webAbsoluteUrl +
                "/_api/Web/Lists/getByTitle('Customers')/items(" + customerId + ")";
 
  // execute AJAX request
  $.ajax({
    url: requestUri,
    type: "GET",
    headers: { "ACCEPT": "application/json;odata=verbose" },
    success: function(data){
      // got the eTag
      var etag = data.d.etag;
  
      var requestUri = _spPageContextInfo.webAbsoluteUrl +
                "/_api/Web/Lists/getByTitle('Customers')/items(" + customerId + ")";
      
      // set the MERGE verb so we only need to provide the update delta
      var requestHeaders = {
        "ACCEPT": "application/json;odata=verbose",
        "X-RequestDigest": $("#__REQUESTDIGEST").val(),
        "X-HTTP-Method": "MERGE",
        "If-Match": etag
      }
  
      var customerData = {
        __metadata: { "type": "SP.Data.CustomersListItem" },
        FirstName: firstName
      };
  
      requestBody = JSON.stringify(customerData);
  
      $.ajax({
        url: requestUri,
        type: "POST",
        contentType: "application/json;odata=verbose",
        headers: requestHeaders,
        data: requestBody,
        success: function(data){
          alert("Updated customer");
        },
        error: function(data){
          alert("Failed to update customer");
        }
      });
    },
    error:  function(data){
      alert("Failed to get customer eTag");
    }
  });
}

Note how the eTag is obtained and set appropriately.

The OData specification provides two verbs for performing updates: MERGE and PUT. Using PUT is impractical as replaces the target object with the object representation that it receives. This means that any properties that are not provided in the representation will be set to null. MERGE on the other hand only updates the properties that are present on the received object representation. This more efficiently utilises bandwidth and is easier to code. However, since MERGE is not one of the verbs that is defined in the HTTP specification, using the MERGE verb may not flow through network intermediaries as seamlessly as methods that are defined in HTTP. As such, the X-HTTP-Method header is set appropriately and we perform a POST. Conceptually, this should be thought of a MERGE rather than a POST operation.

A final note
These examples assume that your code is running in the context of a SharePoint App and is only accessing data in the App Web (or alternatively is running on a page deployed as part of a full trust solution). Should you wish to use the _api in a SharePoint App to access assets outside of the App Web you will need an OAuth enabled solution in order to authenticate beyond the scope of the App. I will undoubtedly blog about this in the future. Please subscribe to my feed to catch that post when it’s written.

RESTful services with jQuery

The goal of this article is to provide a practical understanding of RESTful services with jQuery as it is a fundamental prerequisite to working with the OOTB OData implementation in SharePoint 2013. This is the first part of a series of posts on OData in SharePoint 2013. The other parts are Understanding OData and OData in SharePoint 2013 with jQuery.

For those of you who aren’t familiar with RESTful services, it is important that you first have an understanding of this technology as it’s the basis of the OData protocol. It’s also important to have experience using jQuery to interface with services now that Microsoft advocates this approach. If you already have these skills then you may want to skip ahead to the next post in this series.

jQuery logo - sourced from www.jQuery.com
jQuery logo – sourced from www.jQuery.com

Understanding RESTful services in jQuery

REST (REpresentational State Transfer) is a design model (or architectural style) for web services. Services implementing the REST design model are considered as being ‘RESTful’. These services are accessed via HTTP URIs (as opposed to complex mechanisms like SOAP) and provide a stateless, human readable and structured way of accessing data. REST services can be utilized from a web client with only fundamental knowledge of JavaScript and JSON (or XML) and as such are inherently cache-able  These services can support CRUD (Create Read Update Delete) by leveraging well known HTTP methods – GET, POST, PUT and DELETE – and hence limit the chance of being interferred with by a firewall. So that’s all lovely, but what does it look like?

That was an example of a call to a RESTful service. We will go over it piece by piece but before that I should start by making it abundantly clear that this is jQuery – you must have included a script reference to the jQuery library. Of course you could go back to basics and implement a solution in raw JavaScript (or many other client-side technologies) if you have a specific requirement. I would think very hard before making that decision. JQuery has been adopted by Microsoft and is now included as part of many Visual Studio item templates (including SP Apps and Office 2013 Apps) and greatly improves developer efficiency as well as largely mitigating browser compatibility issues.

The $.ajax is the magic method, but before we call it we have to define an object which represents the settings as to how the ajax call should be performed.

The url property
Of primary importance is the url property. This must identify an endpoint capable of handling the HTTP request. The endpoint in this example is an HTTP handler (.ashx) but it could be implemented in one of a number of ways including as a .NET web service (.asmx), a WCF web service (.svc) or any of a number of ways outside of the Microsoft stack. Implementing services is beyond the scope of this post, however HTTP handlers are great place to start for simple services being consumed from within your web applications but you’ll want to look into WCF services for large scale and public facing API services. In theory the endpoint could even be an aspx file but it would be impractical and has a high overhead.

A REST URI is made up of three parts:
[the base URI (the end point file as just discussed)] + [the resource path] + [the query options].
The resource path in this example is ‘/items’ and defines the resource. You’ll see more on this when we get to OData. Lastly, the query string allows us to refine the result set either specifically as in this case (get me the resource with an ID equal to 8) or notionally (e.g. get me resources that have been modified in the last 12 hours).

The type property
The type property defines the action to be performed as outlined next:

  • GET allows us to Read resource representations
  • POST allows us to Create new resources
  • PUT allows us to Update existing resources
  • DELETE allows us to Delete existing resources

In the case of GET and DELETE the URI itself provides enough information to perform the action. For POST and PUT a representation of the resource is required to be sent through the wire. This is where the data property comes into play.

The data property
The data property is quite self-explanatory and as previously mentioned should only be included when performing a create or update activity. Data can be sent in a variety of formats (see below) however I would recommend using a JSON representation as it is extremely lightweight, human-readable, easily created, easily serialised (stringified) and sent, and easily deserialised (parsed) and worked with. JQuery also makes working with XML reasonably straightforward should you decide to go in that direction.

Visual representation of a JSON object - sourced from www.json.org/
Visual representation of a JSON object – sourced from www.json.org

The contentType property
Paired with the data property is the contentType property. It is only required when then data property is present and it tells the service what kind of data to expect. Some services may work without this (especially if they only accept one data type) but it should always be included. Most commonly one would expect this value to represent json or xml but a service could be written to accept pretty much anything. However, accepting the likes of SOAP would erode the goal of a RESTful implementation. The value for this property should be an Internet Media Type. For a full list of all available types see here but it is most likely that the only ones you’ll need are these:

  • application/json
  • application/xml
  • text/html

The accept header property
The properties discussed so far are all defining the request. The remaining properties define the response (or at least how to handle the response). Defining the accept header lets the service know what kind of data you expect it to return. It should be noted that the service must take the responsibility of reading the accept header and generating the appropriate output. There is no inherent transformation occurring based on this value and services may only support limited response types.

The dataType property
The dataType property defines the type of data that is returned. This value will determine the transformation that is applied to the data returned from the service. If the service returns a stringified JSON object then a dataType value of ‘json’ will mean that the success callback method will receive a JSON object as the returned value. A value of ‘xml’ will return an xml document that can then be parsed easily using jQuery. Other values include ‘text’, ‘html’ and ‘script’ and can be further investigated via the jQuery API documentation.

The callback properties
Finally there are the two callback functions: one for success and one for when an error occurs. Without the success callback a GET call is pointless as it is here that you would define the action to take with the service data (formatted according to dataType). You may decide to omit the success callback for a POST, PUT or DELETE if you don’t need user feedback (but this is very unlikely). Hopefully you understand the premise of callbacks because I’m not going to explain these further (if not, you’ll have no trouble finding good references elsewhere on the web). If you want to find out more about the $.ajax function in general (not specific to REST) try visiting the jQuery API documentation here.

A couple of tips:

  • When testing with any web services using jQuery, make sure you run $.ajax({cache: false}); first to prevent caching responses.
  • You can type GET requests straight into your browser to view the xml/json response for ease of testing. However, you need to prevent the browser from treating it as an RSS feed. In Chrome prepend ‘view-source:’ to the URI (eg view-source:http://hostname) and in IE go: Tools > Internet Options > Content > Feeds and Web Slices > Settings > uncheck ‘Turn on feed reading view’
An example of viewing a service response in IE
An example of viewing a service response in IE

Please read the successors to this article, Understanding OData and OData in SharePoint 2013 with jQuery.

If you’ve found this useful then please subscribe to blog feed.