Integrate with Python based solutions using SharePoint 2016 REST API

Today, I worked with my customer on a project where we want to create list items in SharePoint from a Python based application.

This was my first step into Python and until today I hadn’t used Python before. Luckily Patrick Heneghan understood the Python bits.

Python and SharePoint

We started by just making an REST API call to get information out of our SharePoint 2016 environment. We sorted out the authentication we used the ntlm libraries available for Python.

This resulted into the following script:

import requests
from requests_ntlm import HttpNtlmAuth

auth = HttpNtlmAuth('domain\\username','password')

r = requests.get(
"http://intranet-uat-gb.mysp2016.com/clients/_api/Web/Lists(guid'e28e53b3-f337-44d3-97da-2ea3c64f1221')/Items(4)",
verify=False,
auth=auth,
headers={
'Accept': "application/json",
}
)

This was easy so far. We got the list items and their details as we wanted them from our test list.

Then we wanted to have a look at creating new items and this is where the trouble all started

r = requests.post(
"https://intranet-uat-gb.mysp2016.com/clients/_api/Web/Lists(guid'e28e53b3-f337-44d3-97da-2ea3c64f1221')/Items(4)",
verify=False,
auth=auth,
data={
'__metadata': {
'type': 'SP.Data.TestlistListItem'
},
"Title": "Something better",
}),
headers={
'X-RequestDigest': form_digest,
'content-type': "application/json;odata=verbose",
'Accept': "application/json",
"X-HTTP-Method": "MERGE",
"IF-MATCH": etag,
},

All we would ever get back was the following Invalid JSON message.

-1, Microsoft.SharePoint.Client.InvalidClientQueryExceptionInvalid JSON. A token was not recognized in the JSON content.

Initially I was thinking that token was talking about access token as you find in OAuth authentication, but as we are using NTLM that should not be relevant.

Also our Authentication was working fine as our get methods worked without fail. Then with the help of Waldek Mastykarz we found out that it was something to do with the payload that we were sending. Having made all sorts of changes to the following lines we could not just make the error message go away.

data={
'__metadata': {
'type': 'SP.Data.TestlistListItem'
},
"Title": "Something better",
})

Then I came across a post about Python and SharePoint REST APIs in which the importance of encoding json in Python was mentioned. json.dumps sorts all of this out for us, resulting in the following few lines:

data=json.dumps({
'__metadata': {
'type': 'SP.Data.TestlistListItem'
},
"Title": "Something better",
}),

Now the error goes away and the script is working.  For completeness of this post the foll script as we used is is below. The script will get the 4th item in my test list and update the listitem’s Title to “Something Better”.

import json

import requests
from requests_ntlm import HttpNtlmAuth

auth = HttpNtlmAuth('domain\\username','password')

r = requests.post(
'https://intranet-uat-gb.mysp2016.com/_api/contextinfo',
verify=False,
auth=auth,
headers={
'Accept': "application/json",
},
)
form_digest = r.json().get('FormDigestValue')

r = requests.get(
"http://intranet-uat-gb.mysp2016.com/clients/_api/Web/Lists(guid'e28e53b3-f337-44d3-97da-2ea3c64f1221')/Items(4)",
verify=False,
auth=auth,
headers={
'Accept': "application/json",
}
)
etag = r.json().get('odata.etag')

r = requests.post(
"https://intranet-uat-gb.mysp2016.com/clients/_api/Web/Lists(guid'e28e53b3-f337-44d3-97da-2ea3c64f1221')/Items(4)",
verify=False,
auth=auth,
data=json.dumps({
'__metadata': {
'type': 'SP.Data.TestlistListItem'
},
"Title": "Something better",
}),
headers={
'X-RequestDigest': form_digest,
'content-type': "application/json;odata=verbose",
'Accept': "application/json",
"X-HTTP-Method": "MERGE",
"IF-MATCH": etag,
},
)

print(r)
print(r.text)