SharePoint Online remote authentication (and Doc upload)

The SharePoint REST API is touted as being the tool to provide inter-platform integration with SharePoint Online. However, outside of .NET the authentication piece is not so straightforward. App authentication solves this issue for registered apps but I want to show how remote user authentication can be achieved, regardless of platform.

In a .NET environment please refer to the ADAL library for authentication rather than writing it yourself.

The goal of this post is to provide examples of the HTTP requests which need to be made in order to authenticate SharePoint Online. It then provides an example of using the same technique to upload a document and update metadata just to prove it all works 🙂

The type of applications where this kind of approach may be necessary include: a Java application, a PHP application, or JavaScript application where there is otherwise no SharePoint Online authentication context and the decision has been made (for whatever reason) that user authentication is most appropriate (as opposed to app authentication).

Edit: This approach will not work in a JavaScript environment due to cross-domain restrictions enforced by browsers (unless of course you are on the same domain, in which case you don’t need to worry about any of this anyway). The ADAL.js library is available for the cross-domain JS scenario. I have posted an example here: https://paulryan.com.au/2015/unified-api-adal/

I wrote about using the SharePoint REST API here (and background here, and here).

I will be providing examples of the requests using the ‘Advanced REST Client’ Google Chrome extension.

Authenticate

The authentication piece comes in a few steps:

  • Get the security token
  • Get the access token
  • Get the request digest

Get the security token

First we must provide a username and password of a user with Contribute access to the Roster Data library and the URL at which we want access to the SharePoint Online Security Token Service.

This is done by POSTing the following XML as the request body to:
https://login.microsoftonline.com/extSTS.srf

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope"
      xmlns:a="http://www.w3.org/2005/08/addressing"
      xmlns:u="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
  <s:Header>
    <a:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>
    <a:ReplyTo>
      <a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address>
    </a:ReplyTo>
    <a:To s:mustUnderstand="1">https://login.microsoftonline.com/extSTS.srf</a:To>
    <o:Security s:mustUnderstand="1"
       xmlns:o="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd">
      <o:UsernameToken>
        <o:Username>[username]</o:Username>
        <o:Password>[password]</o:Password>
      </o:UsernameToken>
    </o:Security>
  </s:Header>
  <s:Body>
    <t:RequestSecurityToken xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">
      <wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
        <a:EndpointReference>
          <a:Address>[endpoint]</a:Address>
        </a:EndpointReference>
      </wsp:AppliesTo>
      <t:KeyType>http://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</t:KeyType>
      <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>
      <t:TokenType>urn:oasis:names:tc:SAML:1.0:assertion</t:TokenType>
    </t:RequestSecurityToken>
  </s:Body>
</s:Envelope>

Requesting the security token
Requesting the security token

The response from the request includes the security token needed to get the access token. It is the value which has been stricken out in orange in the image below.

Response including the security token
Response including the security token

Get the access token

Once the security token has been retrieved it must be used to fetch the access token. The can be done by POSTing to the following URL with the security token as the request body:
https://yourdomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0

Request to fetch the access token, passing the security token
Request to fetch the access token, passing the security token

The response from this request includes couple of cookies which must be passed as headers with all future requests. They are marked with the ‘Set-Cookie’ header. We need the ones beginning with rtFa= and FedAuth=. They can be seen the below image of the response headers.

Response includes the access token cookies
Response includes the access token cookies

Get the request digest

The request digest is a .NET security feature that ensures any update requests are coming from a single session. It must also be included with any POST requests.

The request digest is fetched by POSTing to: https://yourdomain.sharepoint.com/_api/contextinfo
The access token cookies must be included as Cookie headers with the request as shown in the image below.

Request to fetch the request digest, passing access tokens
Request to fetch the request digest, passing access tokens

The response from the request will include the request digest in the XML response as in the image below. The entire contents of the FormDigestValue tag will required, including the date time portion and timezone offset (-0000).

Response containing the request digest value
Response containing the request digest value

Upload a document with metadata

Upload the document

Now that we have all the authentication headers we can make update calls into SharePoint Online as the user whose credentials we originally supplied when fetching the security token.

In order to upload a document perform the following POST request:
https://yourdomain.sharepoint.com/subweb/_api/web/lists/getbytitle(‘list name’)
/rootfolder/files/add(url='filename.csv',overwrite=true)

A number of headers must be send with the request including the access token cookies, the request digest (X-RequestDigest) and the accept header as shown in the image below. The body of the request must contain the content of the document being uploaded.

Request to upload a document to SharePoint Online
Request to upload a document to SharePoint Online

The response of this request contains some minimal metadata about the file and can be safely ignored. However, for completeness here it is.

Response JSON from the document upload request
Response JSON from the document upload request

The unique ID property could be used to fetch the document in order to perform metadata updates rather than URL as done in the following example.

Update document metadata

The final step which needs to take place is update the document in SharePoint with the relevant metadata.

This can be done with yet another POST request. This time to the following URI:
https://yourdomain.sharepoint.com/subweb/_api/web/lists/getbytitle(‘listTitle')
/rootfolder/files/getbyurl(url='serverRelFileUrl')/listitemallfields/validateupdatelistitem

All the headers sent with the previous request must be sent with this request as well. The request body is a JSON object which defines the metadata fields to be updated. The fieldname and fieldValue properties must be updated as required. Note that the fieldname property must be equal to the field internal name not the field display name. An example of this is in the image below.

Request to set metadata on a document in SharePoint
Request to set metadata on a document in SharePoint

The response from this request provides success notification for each individual field update as shown below.

Response from the document metadata update request
Response from the document metadata update request

So, this should now be enough to write an application in any server-side language which supports web requests and work against SharePoint Online. I’d love to see some implementations of this, please comment if you’ve done it.

I’d like to acknowledge the following posts as they were invaluable references:

 

Published by

Paul Ryan

As a developer my professional interests are technical and tend to be SharePoint focused. I've been working with SharePoint since 2009 and hope my posts will give back a little to the community that's supported me over this time. I'm also a keen runner (half-marathon) and passionate Brompton bicycle owner.

110 thoughts on “SharePoint Online remote authentication (and Doc upload)”

  1. When I try to generate the the “Request token” I am not getting the response in xml format . I am only getting “Response does not contain any data.”
    And which one is the correct url to use ? Your blog post reads “https://yourdomain.sharepoint.com/_api/contentinfo” and the image shows “https://yourdomain.sharepoint.com/_api/contextinfo”

    Reply

    1. Hi Varun,

      The corrent URI to fetch the request token is: “https://yourdomain.sharepoint.com/_api/contextinfo”. Apologies for the typo, I have fixed it.

      If you are getting a response that doesn’t contain any data when using the correct URI, I suspect that the issue is due to authentication. If you have the access token then you should be able to perform GET requests without the request token to test this (the request token is only required for POST requests (technically non-GET requests).

  2. Hello Paul
    Thanks for your comments. Now we are trying to get the contents of the sharepoint list, through ajax calls, but no success, when I try to show the contents in a java script alert, I am getting a blank alert. Could you please help me with this ?

    Reply

  4. I too have a problem with the contextinfo. I get a 403 forbidden, I have administrator.

    Oddly, I get the same steps to work on my private sharepoint and I get theses steps to work on the sharepoint online when I do HTTP posting in objective C in iOS.

    Maybe a problem with the REST client app?

    Reply

    1. Try adding the following header to your request:

      endpointRequest.Headers.Add(“X-FORMS_BASED_AUTH_ACCEPTED: f”);

  5. I receive the following in the response object:

    “Direct login to WLID is not allowed for this federated namespace”

    Any suggestions

    Reply

    2. I am also facing same issue, i Guess some configuration changes are to be done at server side.
      Please share if you found the solution for this.
      1. Manually registering app to Azure.
      URL1: http://sharepoint.stackexchange.com/questions/122187/authentication-using-sharepoint-online-rest-api-to-create-task-for-organizationa

      2. Enabling SharePoint to be able to receive external Active Directory – ADFS authentication providers.
      URL2 : https://social.msdn.microsoft.com/Forums/office/en-US/c113732d-5e4d-4670-9d40-2a92350e0bc3/direct-login-to-wlid-is-not-allowed-for-this-federated-namespace-error?forum=sharepointdevelopment

  6. Sir
    I have been building a cordova application, with the sharepoint remote authentication(rest api). Login (sharepoint authentication)seems to be working on android phones running those 4.x.x operating system. When I try to use the same code and deploy it in an android device/emulator which is running lollipop os, it did not work and it is neither working on windows phone 8 (IE 11).
    This is the error I am getting POST https://login.microsoftonline.com/extSTS.srf net::ERR_NAME_NOT_RESOLVED. What could be the reason for it ?

    Reply

  8. @slowsword. I am getting a 403 Forbidden when POSTing to get the contextinfo endpoint as well. I am getting the COOKIE values in the previous call with the “rtFA=” and “FedAuth=”

    Have you found any solution to this issue?

    Reply

  9. Hi Paul
    Your article has helped me get further than ever before with sharepoint authentication,
    what would the process be to retrieve lists? i am trying to use the same process as your steps to upload a document but using a GET instead as well as modifying the url to “http://yoursite.sharepoint.com/_api/web/lists/getbytitle(‘listname’)” but i get a 403 forbidden error and X-Msdavext_error: 917656; Access+denied.+Before+opening+files+in+this+location%2c+you+must+first+browse+to+the+web+site+and+select+the+option+to+login+automatically.

    is there some other process that needs to be followed to retrieve and edit lists?

    Reply

  10. Hello paul,

    I’m trying to connect 2 different O365 tenants so I can have 1 calender synced between the two of them. I was hoping to achieve this by the way you propose and the use of REST. Now when I try to get my requestdigest i get a 403 forbidden error.
    The steps to get the cookies worked out fine.
    I did this on an account with full permissions so that can’t be the problem.

    Thank you,
    Lschockaert

    Reply

    1. I imagine the issue is cross-domain related. Browser security restricts you from POSTing from one domain to another. You will need to achieve this by running the code server-side, or by using the SP.WebProxy class that is provided with a SharePoint Hosted app. Good luck – let me know how you get on with this.

  11. I am using Java to connect to Office 365 so I am not sure if this will apply to all of the other 403 Forbidden errors. But I was facing the same issue when I included the X-RequestDigest and the two cookies. The solution to my problem was that I needed to use “Authorization” : “Bearer [RequestDigest]” instead of using the X-RequestDigest header. For eg. In Java this is what I ended up doing

    postItemObj.addHeader(“Authorization”, “BEARER “+moFormReqDigest);

    I hope this helps someone else !

    Reply

  12. So, I was able to get this solution working for Sharepoint Online. But we have this SSL enabled Sharepoint server which we have within our network, I am not able to authenticate using this same strategy. It uses claims based authentication. I do not understand MS all that well, so any help would be greatly appreciated.

    Reply

    1. The first step in the post is about getting the security token. In the steps provided the security token is obtained from the SharePoint Online security token service (STS). You will need to obtain the STS token from your local farm via its own security token service. Once you have achieved this I would expect the remaining steps to work. You’ll have to do some searching as to how exactly to achieve this as I have only done this against SPO.

  13. Hi,

    I am able to get the security token but when I am trying to post it to get the Access token, I am getting forbidden access error. Looking forward for help.

    Regards,
    Ramakrishna

    Reply

    2. in the header:
      “Authorization” “BEARER_TOKEN=[token]”

      Basically, replace “t=token” details with “BEARER_TOKEN=token”

  16. Paul,

    I believe you wont be able to get the digest by javascript if you’re outside of Sharepoint. That’s because if “Origin” parameter it’s included on the request to “_api/contextinfo” then it will fail with 403 forbidden. If you are on Fiddler or something it’s easy because you can delete that header. This applies also to Apache Cordova (unless some weird plugin use)
    Regards

    Reply

  17. Hello, I was reading your article and am having trouble with the same error (403 Forbidden) .I have an application in cord which can authenticate to the online sharepoint and even consume OData. But the cunsumir a REST api to perform a precise Update the RequestDigest, there begins my problems. I can not make a post to ‘/ _api / contextinfo’. Could you help me in this matter?

    $.support.cors = true; // enable cross-domain query
    $.ajax({
    type: ‘POST’,
    data: oAuth.SecurityToken,
    crossDomain: true, // had no effect, see support.cors above
    contentType: ‘text/xml; charset=”utf-8″‘,
    url: oAuth.ProjectURL + ‘/_api/contextinfo’,
    dataType: ‘xml’,
    success: function (data, textStatus, result) {
    var teste = result;

    //digest = $(result.responseText).find(“d\\:FormDigestValue”).text();

    },
    error: function (result, textStatus, errorThrown) {
    var response = JSON.parse(result.responseText);
    if ((response.error != undefined) && (response.error.message != undefined)) {
    alert(response.error.message.value);
    }
    }
    });

    Reply

    1. It looks like you are attempting to perform a cross-domain request from JavaScript. This is an action which is forbidden by the browser (to mitigate XSS attacks). I can see that you are attempting to support CORS to get around this. I think that your issue is that some more steps are required to take advantage of CORS. I have only used it in the context of Office 365 apps with the use of the ADAL.js library. You can find an example of this here: https://paulryan.com.au/2015/unified-api-adal/

      If you are using the ADAL.js library then you can probably skip this step altogether as ADAL.js handles the auth piece for you. Good luck.

  18. Hi,
    I am using OAuth for authentication from my web server so the user is sent to SharePoint Online for authentication and redirected back to my web app once authentication is finished. Is there a way to do OAuth from a native mobile app rather than sending the username and password ?
    Thanks,
    Pranava

    Reply

  19. Hi,
    I got message in Body “No Response Received” after “Get the request Digest” Section.

    I have sent FedAuth & rtFa as request Header.

    Please help me out…
    Thanks In Advance…!

    Reply

    1. The first thing to note is that this approach is for authenticating to SharePoint Online, this won’t work for an on-premises SharePoint environment.
      Beyond that, I wouldn’t have thought that it would make a difference if you have synch’d users from ADFS but I really don’t know – I haven’t done it. Sorry I can’t be of more service 🙁

  21. Alright, thanks for replying though. This article helped me more than anything else out there.

    On a trial vanilla O365 environment I can make this fly, but on a company with custom authentication for their O365 tenant I now get “Direct login to WLID is not allowed for this federated namespace”. There are quite a few hits on it out there, I hope to solve it soon 🙂

    N.B: Not sure if this will start a new thread, couldn’t find a “Reply” to your reply, if that makes sense.

    Reply

  22. Hello Paul,

    I was following your steps to get the accessToken. I was able to get the security token but when I sent the security token as the request body to the url mentioned to get the access token, I got a response whose headers did not contain the COOKIE values with “rtFA=” and “FedAuth=”. Could you possibly help me with this ?

    Reply

    2. Hi Anirudh/Paul,
      I was able to get the security token, but when I try to use it to get the access token, I get a 500 Internal Server Error and occassionally a success message but with the cookie headers not present.

      Any idea what might be the cause. I double checked the URL (https://mydomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0) and my security token is t=EwAwA06hBwAUNfDkMme61kIdXqvj9tWnUbHtXWEAAa9UbHrnSu2TW8tTiuJs79C62yOGk2sNgSx+zNXiWMI9m0FhZZaTPZX2Pfp64MUahMgnFYcmUckgkgoF29z1RKkIDtoDJG6jXQu8RCUWljZNFTjbcLhAkWV0F4RUruALM4hJbVxWhAZvuHM+DjbUKiiKRXWHzDBlzNgQBXjTn2Yb62505iNsgmym6arkJSGmE4kPlBT8gh2r2IiaTBClj2wBFKQ0f+c9tfjPdBliAGxb+NfoX48V+zL79eCYNFMJ22ZUeai/I81yHbfvmxohHmIvH56B6lXOPwMAOzPENLEuI/IwmJc176ylUXg/y5GYte3vf04zlDGrTjIhItJjyIwDZgAACPn3EmHGnPBnAAKkRj+KFyoRnQL7lqtC9/bQJ83PiorciVZLiqHaq1P16H+gqcu9s9JxBBjY1spI64Q+xn4FPt/m1DMhvFUNHiwDYATYpC8eBu11UWoxKTMbYeu93ljShvmhb+sltkpClhEYv13KvETd38RQqS6C+bOuJ7j/ncB4aRP3k5EehmDDV101GYrfsg/ArLEObyBVp6QaiDZ0GraaBiY1OMmItTOMm8Q5c9Z/bki3d5FhIO32yNHhRSYP2mNtdo2Adb/chtZD7LKeq7129wQANYowlwXH1jQ2Ns6HRLXpmcrtDJxe5mfSc+s1xdlJBVwfVXJhdV12HBkCfPjlylANI38rWXrlRCTKJpV3h4jckLvq10m4SXrti605B3WLtRSFjnZa19gBwD8dA3BsyI6Ad4bFrAARzOiBHbt5kL4sf6vEwjChyzL0s4sxMcqoVwJg36YjabsJfJfixhK9ohd+luwdYWLGH7HTn12dwCeBWqb7KXTjSZxlXHWk38xyO7FPPnkl/pTO4B+HUepppluCtTVCRoF87omoe6cYueZuBl0Dy8E1RgjrEDAaQ7hrUtTPVbv/6xR8rjrYrWizczmNhl3PrxECmr+ylYFY1yvJvuObUzRDH1ErH1xIksh3cAhinamMon/5O8n+XE/vB1kAnbnxwaoc+EyO4CoYCPDUBpqoSIEcrVgC&p=

      Anything else I am missing?

  24. Hello Paul,

    Is there a way to make all these steps run using jQuery or client side Javascript ? I have been trying to get these steps automated but I am having issue with CORS.
    If not for CORS, my code runs flawlessly.

    Please help.

    Thanks & Regards,
    Anirudh.

    Reply

    1. Hi Anirudh,

      A the top of the post there’s a block quote discussing this very point. You’ll want to take advantage of the ADAL.js library. There’s a link to another post I have done regarding this.

      Paul

  25. Hi Paul,
    I used the same authentication mechnaism to get the cookie string and get the folders and upload the file for onedrive for business.
    But it is used to work perfectly till last week.
    But now I get the below error

    403
    {“error”:{“code”:”accessDenied”,”message”:”The caller does not have permission to perform the action.”}}

    I am not sure the reason for this issue
    Pelase cna ou help.

    Reply

  26. Dear Paul,

    This is very useful, thank you very much.
    Our App uses O365 REST API with OAuth2 access tokens. Can we use the same tokens to access Sharepoint or do we have to perform a new authentification?

    Thank you

    Reply

    1. Your tokens are valid per domain. So assuming you have access tokens for graph.microsoft.com then no, you can’t use them to access tenant.sharepoint.com. Before the introduction of the Microsoft Graph API (previously know as the unified API) you would need to fetch access token for each service – Mail, Calendar, OneDrive, etc – and this is the exact issue that the unified API was introduced to solve.
      Potentially SharePoint data (at least search?) will be available via the Microsoft Graph, but for now you will need a separate token for each.

  28. Any idea how this would work for on prem SharePoint? I’m guessing that rather than asking for a request token from local.microsoft.com, we’d have to POST the local STS. Have you had experience with that?

    Reply
  29. Pingback: Authenticate to SharePoint REST API from external service » Rent in London

  30. Anyone getting a 403 error with /_api/contextinfo in SharePoint Online, make sure the cookie is set correctly in your header. I was able to get this to work in Fiddler with the following format:

    Cookie: rtFa=[insert your token]; domain=sharepoint.com; FedAuth=[insert your token]; path=/; secure; HttpOnly:

    Reply

    2. Confirmed this fixed the 403 error for me at the contextinfo step.
      I am using Advanced REST client.
      Kept getting stuck here and finally scrolled far enough through the comments to find this post.
      TYVM Kirsty

  31. Hello and Thanks for this article.

    I was able to get the list of folders from sharepoin but when I try to add a new Folder, I get a strange error, can you suggest what is happening?

    Error:
    {
    “error”: {
    “code”: “-2130575251, System.Runtime.InteropServices.COMException”,
    “message”: {
    “lang”: “en-US”,
    “value”: “The security validation for this page is invalid. Click Back in your Web browser, refresh the page, and try your operation again.”
    }
    }
    }

    My POST REQUEST HEADERS:
    Cookie: FedAuth=; rtFa=
    X-RequestDigest:
    content-type: application/json;odata=verbose
    accept: application/json;odata=verbose

    POST https://site/teams/sn-integration-test/_api/Web/Folders

    Payload:
    {‘__metadata’:{‘type’:’SP.Folder’},’ServerRelativeUrl’:’/teams/sn-integration-test/Shared%20Documents/Change%20Record%20Documentation/test1234′}

    Reply

    1. This is the error you get if you haven’t provided the request digest. You need to provide this with POST requests but not GET requests so it makes sense that this is your issue. Please ensure that you are passing this header correctly with your request.

  32. Hi Ryan,

    Followed all the above steps mentioned by you and was able to successfully achieve it.

    Created a folder successfully through postman only.

    But unable to upload a file to a libary as you did, could you please help me with some snippets on uploading files from local machine to library . unable to construct the url and request body.

    ex:-
    url :- https://hostname/site/_api/web/lists/getbytitle('POC%20Library‘)/files/add(url=’C:\Users\Desktop\2.jpg’,overwrite=true)

    method :post

    response error: The parameter name url is not valid.

    Reply

  33. in continuation for the above comment.

    How to upload a file consisting some content text from a local drive to sharepoint online using rest api through postman.

    Need detail steps please.
    along with request body and headers.

    Also is there any way that we can run javascript/jquery in postman so that it would be easy for us to stream a file and upload.

    Reply

  34. Hi ,
    I am using the same api for getting the token for one drive for business. Earlier it used to work. But now it fails and I get the cookie “”.

    Please suggest if anyone having the same issue.

    Thanks and best regard
    GP

    Reply

  35. Be aware that SPO made a change on 12/2 that is more strict about the handling of the request to /_forms/default.aspx, and if you specify a content-type other than Content-Type: application/x-www-form-urlencoded
    it will fail. Some libraries use some other Content-Type as the default, I saw this in a Java lib. Make sure to specify the content type above and it should still work.

    Reply

  37. Hello Paul:

    Thank you for your post, I haven’t been able to make it work tough, I did this:
    1.- Get security token, done, no problems with that.
    2.- Get access token, I can get the cookies on the response.
    3.- Get request digest, with
    https://mysite.sharepoint.com/_api/contextinfo
    I set a couple of Cookie headers, and I keep getting 403.
    I can not mode forward with that, cookies I get are in this form:
    rtFa=[long_string]; domain=sharepoint.com; path=/; secure; HttpOnly
    FedAuth=[long string]; path=/; secure; HttpOnly
    In your images is not clear if you use the whole thing or only the long string part, I tried with different combinations and always get 403. I’m using JAVA for my tests. Any feedback would be so much appreciated.

    Reply

    1. It appears at though you are doing the right thing. The cookie values are the entire value of the Set-Cookie response header. I think that the most likely issue is that you aren’t correctly setting the headers in your Java code. As I’m not a Java dev I won’t attempt to provide a code sample for this but I suggest getting it working in the browser first, e.g. using the REST client I demonstrate using, or Postman, or Fiddler. Good luck.

    2. Hi,

      Did you get it any way working by any chance ? I am looking for some code reference on JAVA/Apringboot. We are using springboot to write data to a remote sharpoint and also need to connect for download using restapi backchannel call.

  38. I am trying to write a powershell script that automatically uploads logs files from a computer to a remote sharepoint site. I am having trouble uploading the file. I have been able to authenticate using PKI soft certificates to the remote sharepoint site and download a webppage successfully by passing the fedAUTH value in a cookie to the sharepoint site. But, any attempt to upload or download a file from the Shared Documents folder is unsuccessful. Any help would be appreciated.

    Reply

  39. Hi Paul.

    Very helpful guide by the way.

    I was able to get the cookies from the header when I started on this earlier this year, but now the query (in POSTMAN, with STS Token as RAW body data) returns the below:

    https://MyDomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0

    Cache-Control →private, max-age=0
    Content-Encoding →gzip
    Content-Length →47283
    Content-Type →text/html; charset=utf-8
    Date →Tue, 28 Feb 2017 17:53:38 GMT
    Expires →Mon, 13 Feb 2017 17:53:39 GMT
    Last-Modified →Tue, 28 Feb 2017 17:53:39 GMT
    MicrosoftSharePointTeamServices →16.0.0.6216
    P3P →CP=”ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI”
    SPIisLatency →1
    SPRequestDuration →544
    SPRequestGuid →8080d99d-00a1-3000-b1d6-3049c21273ae
    Server →Microsoft-IIS/8.5
    Strict-Transport-Security →max-age=31536000
    Vary →Accept-Encoding
    X-AspNet-Version →4.0.30319
    X-Content-Type-Options →nosniff
    X-FRAME-OPTIONS →SAMEORIGIN
    X-MS-InvokeApp →1; RequireReadOnly
    X-Powered-By →ASP.NET
    X-SharePointHealthScore →0
    request-id →8080d99d-00a1-3000-b1d6-3049c21273ae

    I have searched for a long time, but alas cannot find if MS have changed this process, could this be possible or have i made a mistake?

    Reply

  40. Hi Paul,
    thank you very much for your tutorial! It was really helpful!
    So my application is working now with getting lists using the cookies FedAuth and rtFa (don’t need the request digest). My problem is, that I don’t know when the values of the cookies will expire and if I have to refresh them (and how).
    I found this: https://support.office.com/en-us/article/Session-timeouts-for-Office-365-37a5c116-5b07-4f70-8333-5b86fd2c3c40
    Which Office 365 service is the correct one of the authorization you described? Sharepoint Online or Azure Active Directory?
    And is there a possibility either to refresh easily my cookie values instead of doing the whole process or to set that my authentication will not expire?

    Reply

    1. Hi James,

      I am trying to use the solution provided by you into Linux environment for retrieving the list data from sharePoint using the Get method and getting below error message

      “curl: (7) couldn’t connect to host
      ./sharPoint_upload.sh: line 31: rstoken: No such file or directory”

      We are struggling with the issue since last one week, any help will be greatly appreciated.

  43. Hi, let’s say i have a job repeatedly talking to SharePoint. Currently the user account it uses to connect to SharePoint is in a configuration file. Is it safer to store an authentication cookie in the configuration file instead? Is there a way to keep the authentication cookie alive?
    Some online services like the powerBI portal require you to login, and after that point the service is able to talk to SharePoint. Do they just store an authentication cookie? Or are they actually storing the username and password somewhere? Please let me know your thoughts and ideas around this.

    Reply

    1. Hi, you should be using app/add-in authentication rather than user authentication. Do not store authentication cookies in persistent storage. They have a limited lifespan and will eventually fail. App/add-in authentication can be achieved with SharePoint in two ways: as a SharePoint app or as an Office 365 app (in the case of SharePoint Online). Using app authentication the job can have fine grained permissions to achieve the given job without the risk of privileged credentials being leaked.
      In the case of app authentication you’ll be storing a Client ID (aka App Id) which is essentially the app’s username. In your case where the app does not require a user presence, you will also be storing a Client Secret (aka App Key) which is essentially the app’s password. App/add-in only authentication certainly has some complexities involved (you’ll likely need to create self-signed certificates) but is the correct way to achieve this.

      https://dev.office.com/sharepoint/docs/sp-add-ins/authorization-and-authentication-of-sharepoint-add-ins

  44. Has anyone solved the 403 forbidden response from the post to https://mysite.sharepoint.com/_api/contextinfo ? I am using Postman. I can get the security token, and then get the cookies in the get access token step. In my browser the FedAuth and rtFa cookies are set. I can do a query such as getting the sub folders from a parent folder, so I am authenticated to the site. But I cannot get a 200 response from contextinfo. Could this be a permissions issue? If so, what specific permission should be enabled? Thanks!

    Reply

    1. contextinfo requires that you POST rather that GET. Are you doing this? Try using a user with Owner permissions the site to rule our permission issues. Otherwise it sounds like you might just not be passing the cookies with the request correctly.

  45. Thanks for the response, Paul! Turns out it was a permissions issue. When I created a new site under my account so that I had Owners permission, everything worked. Thanks again!

    Reply

    2. The steps provided in the post are agnositic of the client executing the HTTP requests. I can only assume that that something has changed during the migration which has impacted how the steps in the post are being executed.
      In the case of a UWP app though, there are much easier ways to achieve authenticaion using the ADAL library provided by Microsoft. Please use a supported library for tasks like this where you can.
      https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-authentication-libraries

  47. Hi paul,
    i know this is somewhat like repeated question but please let me know if anything else is needed to get accesstoken,
    1) https://login.microsoftonline.com/extSTS.srf
    first step i got security token in response
    2) POST https://mydomain.com/_forms/default.aspx?wa=wsignin1.0 HTTP/1.1 1 1 map[Content-Type:[application/x-www-form-urlencoded]] {t=tokenvalue}
    i got 403 Forbidden in response
    i do have the admin privileges also, not sure of what is going wrong here in this request.
    API REQUEST USING POSTMAN

    Reply

  48. Hi Paul Ryan,
    i got the security token in first request and when i sent security token to get cookies,
    POST https://mydomain.sharepoint.com/_forms/default.aspx?wa=wsignin1.0 HTTP/1.1 1 1 map[Host:[mydomain.sharepoint.com] Content-Type:[application/x-www-form-urlencoded] User-Agent:[Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36]] {tokenvalue}
    Response:
    msg=”response cookies: [buid=AQABAAEAAABHh4kmS_aKT5XrjzxRAtHz4CnuC62XDq4sJ3SeA5IcTg_k_MQwmf1Q-9KI8xLD3jAyvDqks1IgqAqIuSFuyPxdlKeZTUCjo_wAGKDv5QXoBR_59M26yPfvdH_0XzM4ZAMgAA; Path=/; Expires=Fri, 22 Dec 2017 09:52:27 GMT; HttpOnly; Secure esctx=AQABAAAAAABHh4kmS_aKT5XrjzxRAtHz4TpIsdAVG0ZDIRQbjTLTLT4RLNgLcle6JtkKZxHNvLpwT2Judw9wpWA6otlLnnCIZ37WII71Rx4_Du4xt9r8hEgWR2TWrmS7__x_Jf4C3qP9eJRP_ryLRTwcwrH9kgpqXabJx20rPf1glX-XyQCvCoutTUJrkSVxTTUN2yO2pCggAA; Path=/; Domain=login.microsoftonline.com; HttpOnly; Secure x-ms-gateway-slice=004; Path=/; HttpOnly; Secure stsservicecookie=ests; Path=/; HttpOnly; Secure]”
    i did not get fedauth and rtfa cookies in response.

    Reply

    1. When you get a 302 redirect, this is normal. If you do the request in Postman, it will automatically redirect and add the cookies to the request (unless you turn this off). In the 302 response is the cookies, no need to visit the redirected location.

  50. I know this article is a few years old now, but I also had what seems like a common 403 problem mentioned in the comments and wanted to share how I resolved it.

    This article suggests getting the request digest by posting to https://yourdomain.sharepoint.com/_api/contextinfo. When I did this to get the digest, then tried to use that digest to add a document into a subsite like yourdomain.sharepoint.com/sites/yoursite, I would get a 403. I suppose this may be because the subsite is part of a different site collection from the root. Instead, when I posted to https://yourdomain.sharepoint.com/sites/yoursite/_api/contextinfo to get the digest, the request digest I received then worked fine in the subsequent step of posting to add a document to a library. The document was uploaded just fine.

    Reply

  51. I’m getting below error.
    ————————–
    Did not understand “MustUnderstand” header(s):{http://www.w3.org/2005/08/addressing}Action, {http://www.w3.org/2005/08/addressing}To, {http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}Security
    —————————–
    can you please help

    Reply

  52. Hey, I’m trying to login to sharepoint and this solution is not working anymore. I searched the web and it seems that something changed in august 2018. Do you know some other solution to this?
    Thanks

    Reply

  54. Any ideas on how to do this if the domain you are working with uses login.microsoftonline.com then goes to Centrify to log in? When I do the request to get the Security Token it keeps telling me I have an Invalid Username or Password and I think the Centrify thing is my issue.

    Reply

  55. To get security token, we are passing user credential in plaintext in request headers. Though it’s HTTPS and POST method, still would it not be security risk? anyone with wireshark/tcpdump/fiddler softwares can intrude and read this credential. Are we supposed to use encrypted values for username and password (not sure SPO will authenticate this credential)? Please suggest security solution for it.

    Reply

  56. Hi,

    I stuck at first step. I tried to authentication and in response i get
    “No tenant-identifying information found in either the request or implied by any provided credentials.”
    can you please suggest what I need to do?




    S:Sender

    wst:FailedAuthentication


    Authentication Failure

    0x80048821

    0x80048821
    AADSTS50059: No tenant-identifying information found in either the request or implied by any provided credentials.

    Reply

  57. How this solution will behave in case of multi-factor authentication? I am getting Authentication failure message when trying to get security token.
    Thanks

    Reply

  58. I tried this solution but getting a Forbidden:403 error at the step of get the request digest with providing both rtFa and FedAuth Cookies in the header. Don’t know why? Is it related to the comments that Adriel mentioned earlier?

    Reply

  59. i am getting below error

    {Transfer-Encoding: chunked
    X-SharePointHealthScore: 1
    X-Forms_Based_Auth_Required: https://cnhcorg.sharepoint.com/_forms/default.aspx?ReturnUrl=/_layouts/15/error.aspx&Source=%2f_vti_bin%2fclient.svc%2fweb%2fGetFolderByServerRelativeUrl(%27%2femail%2f_6e07afbf-a945-e911-a993-00224800c4f1%27)%2fFiles%2fadd(url%3d%27Certificate.pdf%27%2c%2520overwrite%3dtrue)
    X-Forms_Based_Auth_Return_Url: https://cnhcorg.sharepoint.com/_layouts/15/error.aspx
    X-MSDAVEXT_Error: 917656; Access+denied.+Before+opening+files+in+this+location%2c+you+must+first+browse+to+the+web+site+and+select+the+option+to+login+automatically.
    DATASERVICEVERSION: 3.0
    X-IDCRL_AUTH_PARAMS_V1: IDCRL Type=”BPOSIDCRL”, EndPoint=”/sites/CNHCCRMTest/_vti_bin/idcrl.svc/”, RootDomain=”sharepoint.com”, Policy=”MBI”
    SPRequestGuid: 58e9d39e-a0a7-8000-4383-e019b1bf979d
    request-id: 58e9d39e-a0a7-8000-4383-e019b1bf979d
    MS-CV: ntPpWKegAIBDg+AZsb+XnQ.0
    Strict-Transport-Security: max-age=31536000
    X-FRAME-OPTIONS: SAMEORIGIN
    MicrosoftSharePointTeamServices: 16.0.0.8803
    X-Content-Type-Options: nosniff
    X-MS-InvokeApp: 1; RequireReadOnly
    X-MSEdge-Ref: Ref A: 1B8DC2D2B866466AA29589CD9FFF6AEB Ref B: LON21EDGE0413 Ref C: 2019-04-17T17:47:00Z
    Cache-Control: private, max-age=0
    Content-Type: application/json;odata=verbose;charset=utf-8
    Date: Wed, 17 Apr 2019 17:47:00 GMT
    Expires: Tue, 02 Apr 2019 17:47:00 GMT
    Last-Modified: Wed, 17 Apr 2019 17:47:00 GMT
    P3P: CP=”ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI”
    X-AspNet-Version: 4.0.30319
    X-Powered-By: ASP.NET

    }

    Reply

  60. Hi Paul,

    We are using the sharePoint Online and want to retrieve the list data using rest API from linux environment with below curl command but not able to succeed, any guidance into issue will be great help.

    curl -i -X GET – -H ‘siteClientId:cdfd5f77-b45c-4ad6-bb1b-bc50562817cd’ -H ‘siteSecretId:9kG3wP/yEAarjCAbvA4qIPSKnwfqCUog1SYDU2t1D2I=’ ‘https://collaboration.***.com/sites/pocsite/_api/lists/getbytitle(‘EM%20Task%20List’)/items’

    “X-MSDAVEXT_Error: 917656; Access+denied.+Before+opening+files+in+this+location%2c+you+must+first+browse+to+the+web+site+and+select+the+option+to+login+automatically.”

    Reply

  61. Hi Paul,

    I am trying to authenticate a user in a.sharepoint.com domain, i could able to authenticate using the steps provided in your site.

    But the same procedure doesn’t work, when i try to authenticate a user present in a.sharepoint.com domain to b.sharepoint.com domain, the user in a domain have access in b domain.

    Please list the procedures to authenticate the user between cross domains.

    Reply

  62. Hey Paul,
    Great article!! is this possible for external user?
    I have a gmail account which is connect to sharepoint.
    The request for the access token fails with the following message:
    “AADSTS50034: The user account {EmailHidden} does not exist in the gmail.com directory. To sign into this application, the account must be added to the directory.”
    The user is indeed connected to sharepoint and if I login through the browser I can then issue for example the following rest call
    https://mytenant.sharepoint.com/subSite2/_api/web/GetFolderByServerRelativeUrl('/subSite2‘)/Folders?$expand=Properties

    I believe the authentication with the browser is done by oauth2… can you please help here, how to get the token for the external user..

    Appreciate it thanks.

    Reply

  63. Hi! I’m trying to use your sample.
    I can’t use my user and password.
    But, I did generate a clientid and secret.
    Is it posible use this code with clientid and secret?

    Reply

  64. Hey Paul,

    I am getting the below error when I am trying to authenticate. Please guide me further.

    0x8004882c0x80045b00

    Thanks,
    Leela.

    Reply

  65. Hi Paul, By using this method is there any security concerns as the user name and password is not massed in the post pay load ? also is there a way we could do the same by using JWT token with password grant type rather than using client cridentials ?

    Reply

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.