Python Requests Library (HTTP requests)

Are you trying to understand how to use the Python requests library? In this article I will explain how to send POST requests, GET requests, and requests headers, using the Python requests module/library. Moreover, I will explain how you can parse JSON response using the Python Request JSON built-in method.

Introduction

The Python requests library is a small module that allows you to send HTTP requests using Python. It is similar to the native Python module urllib2 (the replacement of urllib2) but it has a more simple syntax. For example, with the Python request module, you do not need to encode any parameters before sending them. Additionally, the Python request module has a built-in method to parse JSON objects, namely json(). Many different kinds of HTTP exists, but we will focus on the two most important ones:

• GET requests
• POST requests

Before you can use any of the mentioned code below, we will need to install the Python request library. If you have installed Python using Anaconda or downloaded it from Python.org, then you will already have pip installed on your device. However, if for some reason you do not have pip installed, please follow these instructions to install pip. You can now run the following code in your terminal to install the Python request module.

pip install requests

Python Requests Get

The GET request is the most simple Python HTTP request you can execute using the Python requests library. This request only requires a URL and optionally a header that adds more information about your request. The screenshot below shows you a simplified diagram of what happens during a GET request.

Within Python, you can send a GET request using the following lines of code. The response variable will now contain the response from your GET request. Do not forget to import the Python requests module by using import requests.

import requests
response = requests.get('https://jsonplaceholder.typicode.com/users/1')

Python Requests Post

The POST request is a bit more complicated Python HTTP request. This request requires a URL and a payload that contains data that you want to “post”. You can also optionally add a header for more information about your request. The diagram below shows a simplified of what happens during a POST request.

Within Python, you can send a POST request using the following lines of code. As you can see, you can send the payload as a Python dictionary object. If you are unfamiliar with Python dictionaries, you can check this article. Also here, do not forget to import the Python requests module by using import requests.

import requests
url = 'https://jsonplaceholder.typicode.com/users/'
payload = {
    "title": "test title",
    "body": "test body"
  }

response = requests.post(url, data = payload)
print(response.text)

Python Requests Headers

Whether you are sending a POST request or a GET request, you are always able to add a header to your request. With the Python requests library, you can do this using the following lines of codes. I have added an example of how to get the status code of a request by using .status_code. You can check the definitions of different status codes on this page.

import requests
url = "https://jsonplaceholder.typicode.com/users/1"
headers = {"user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.150 Safari/537.36"}

response = requests.get(url, headers=headers)
print(response.status_code)

Python Request JSON

As mentioned before, the Python requests library has a built-in module to parse JSON objects. The name JSON is an abbreviation of JavaScript object notation. Normally, you would need to import a module called JSON, before you can work with JSON objects in Python. If you like to know a little bit more about JSON objects, and how they relate to dictionary objects in Python, you can check my other article. With the code below you will parse the JSON response and the resulting type will be a dictionary.

import requests
url = "https://jsonplaceholder.typicode.com/users/1"

response = requests.get(url)
print(type(response.json()))

I hope you now understand how to send POST requests, GET requests, and requests headers, using the Python requests module/library. I have also explained how to parse JSON objects using the Python Requests JSON built-in method. Please let me know if you have any questions in the comments below!