If you just need help to solve your problem you’re best off just reading the code-snippets and avoid the rest of this blog entry which merely describes my motivations behind the code.
So this started when I wanted to use the GitHub API to upload automated builds as releases yesterday. I haven’t used REST APIs much before so I read a little from their documentation and it seemed easy enough.
At first I thought I’d just use cmd or Powershell to do the deed. But the logic seemed a little too complex for cmd and I’d need something like curl. The effort to do it in Powershell seemed like a waste, since I might use this for UNIX systems later too. Since other parts of the build required Python anyway I decided to just do it in Python. I perused the web about the basics to use Python as a REST client. This excellent blog post was a helpful starting point: http://isbullsh.it/2012/06/Rest-api-in-python/ . Although it deals with Python 2 and I had Python 3 installed.
I really resented the suggestion that I’d have to add another dependency for this simple task and thought that Python had enough “batteries included” to just do it with the standard libs. So I researched a bit more about urllib2 (or urllib.request in Python 3) and it seemed to suffice for my use-case. It can’t send anything else than POST or GET requests. So if you need to use DELETE or other http methods you’ll likely need to use the http.client module.
So I clobbered together some basic script to dispatch GET and POST requests for the GitHub API for use with personal access tokens as authentication method (which you can generate here: https://github.com/settings/tokens for your account). This doesn’t work if you use two-factor authentication but rewriting the authentication part to work with OAUTH tokens instead shouldn’t be hard and work with two factor auth.
Anyway, long story short, here’s the snippet (sorry for the terrible formatting, you’ll be better off viewing it on GitHub):
import urllib.request
import base64
import sys
import os
def GitHubRequest(repository, credentials, url, data=None, datatype=None, useRawURL=False ):
""" GitHubRequest(repository, credentials, url, data=None, datatype=None, useRawURL=False ) -> response, returncode
This function is thoroughly unpythonic and you should probably just use it
as a starting point.
It dispatches a request to the GitHub API (written with v3 in mind).
A GET request if data is not specified, a POST request with the string in
data if datatype is not specified and a POST request with the contents of
the filename in data if datatype is specified.
Arguments:
repository - GitHub with in the format 'User/Repository'
credentials - GitHub username and personal access token in the format
'username:accesstoken'
url - the GitHub API url, for example 'issues/3' or a complete
url if useRawURL is set to True
data - string specifying the data to be send, if datatype is
None the string is send, otherwise a file with the name
is opened and sent
datatype - the type of data to be, if it's a str it will be used as
MIME type for the POST request, if it't not a str or
NoneType then the MIME type will be 'application/octet-stream'
useRawURL - specify whether the url is the full request URL (when
True) or just a partial url to append to 'apiurl/repo/'
"""
if isinstance(credentials,str):
credentials = bytes(credentials,'UTF-8')
if useRawURL == True:
requesturl = url
else:
requesturl = "https://api.github.com/repos/%s/%s" % (repository, url)
print("request: %s" % requesturl)
#GET request
if data==None:
req = urllib.request.Request(requesturl)
else: #POST request
if datatype==None:#JSON POST request
req = urllib.request.Request(requesturl, data=bytes(data,'UTF-8'))
else: #File POST request
filehandle = open(data,'rb')
req = urllib.request.Request(requesturl, data=filehandle)
filesize = os.path.getsize(data)
req.add_header("Content-Length", "%d" % filesize)
if isinstance(datatype,str):
req.add_header("Content-Type", datatype)
else:
req.add_header("Content-Type", "application/octet-stream")
if credentials!=None:
base64str = base64.b64encode(credentials)
req.add_header("Authorization", "Basic %s" % base64str.decode("utf-8"))
try:
handle = urllib.request.urlopen(req)
except IOError as e:
code = -1
if hasattr(e,'code'):
code = e.code
message = str()
if hasattr(e,'fp'):
message = e.fp.read()
if 'filehandle' in locals():
filehandle.close()
return message, code
response = handle.read()
if 'filehandle' in locals():
filehandle.close()
return response, handle.getcode()
And here’s my main usage case (similar version better viewable at GitHub):
#!/usr/bin/env python
from GitHubRequest import GitHubRequest
import urllib.request
import base64
import sys
import os
import json
def main():
credentials = bytes(sys.argv[1],'UTF-8') #in the format 'User:privateaccesstoken'
repository = "Bigpet/rpcs3-buildbot-tools"
filename = "some.zip"
releasename = "sometag"
commitish = "3d2659fb20061d43a0057830fca30101c329e06a"
#Check if the tag already exists
response, code = GitHubRequest(repository,credentials,"releases/tags/%s"%releasename)
print("code: %d"%code)
if code == 200: #already a release with this tag there
#expected
code = 200 #do nothing
elif code == 404: #no release with this tag yet
#Create release
requestdict = {'tag_name': releasename, 'prerelease': True}
if commitish != None:
requestdict['target_commitish'] = commitish
response, code = GitHubRequest(repository,credentials,'releases',json.dumps(requestdict))
if code != 201:
print("got unexpected return code %d while creating a release: %s"%(code,response),file=sys.stderr)
sys.exit(1)
else:
print("got unexpected return code %d while looking for release: %s"%(code,response),file=sys.stderr)
sys.exit(1)
#Get upload_url
resdict = json.loads(response.decode('utf-8'))
upload_url = resdict['upload_url']
upload_url = upload_url.replace('{?name}',"?name=%s"%filename)
assets = resdict['assets']
for asset in assets:
if asset["name"]==filename:
print("File %s already exists in tag %s"%(filename,releasename),file=sys.stderr)
sys.exit(1)
#consider just using "application/octet-stream" for generic files
response, code = GitHubRequest(repository,credentials,upload_url,filename,"application/zip",True)
if code != 201:
print("got unexpected return code %d while trying to upload asset to release: %s"%(code,response),file=sys.stderr)
sys.exit(1)
if __name__ == "__main__":
main()