Searching Tweets Using Twitter API

Disclaimer: this is the English version of my work which is also available in Brazilian Portuguese, so, if you prefer, you can access that version by clicking here.

Creating your Developer Account

The Twitter API is a tool provided by Twitter that allows real-time access to what is being tweeted by its users. To access it, you will first need to have a developer account; you will need to request one, click here to go to the request page.

Once you gain access to the Developer Portal, you will notice a dashboard like the one below.

Now to connect to an API endpoint, you will need to create a project and an App. To do it, click on Create New Project and fill in the required information for the project and the App creation. After filling out the entire form, you will see a screen with the authentication keys of your application; they will be necessary when connecting to the API.

Now you have everything needed to connect to the API!

API Connection

It is possible to use several different technologies to create the API connection, but we will use Python and some of its libraries.

First, about the libraries that will be used, there will be three: Python Requests, json and Python Decouple. Python Requests will be responsible for carrying out the HTTP requests, json will help handle the responses obtained for the .json format, and the Python Decouple library will assist in the separation of the authentication from what is in the code.

So, first, we will import the required libraries.

from decouple import config
import requests
import json

After that, we must import the authentication value into the code for that, we will create, in the same directory of our file, a file with the name .env, and inside it, we will add the following information.

BEARERTOKEN=BearerTokenValue

After the file creation, it is now possible to use the Bearer Token authentication value without adding sensitive information to the code. To do this, we'll add the following line of code to our .py file.

bearer_token = config('BEARERTOKEN')

The bearer_token variable will now hold the value of your Bearer Token.

Now, we will create a function that adds authentication in the header of the HTTP request.

def set_auth(ob_auth):
    objeto_para_auth.headers["Authorization"] = f"Bearer {bearer_token}"
    return ob_auth

After that, we can carry out the request process for the API. We will define our search parameters and API endpoint.

search_url = "https://api.twitter.com/2/tweets/search/recent"
str_busca = f'to:{profile} {keyword} lang:{language}'
query_params = {'query': str_aux,'tweet.fields': 'attachments,author_id,created_at,lang,public_metrics,source','max_results':limit_tweets}

In the code above, the search_url variable receives the endpoint that will be used (you can read a little more about the endpoints available by clicking here). Furthermore, the str_busca variable receives the search string in the case of the example the search will be for tweets sent to a profile, containing a search key and written in a specific language, you can found other search filters here. Lastly, the query-params receives the search string, the fields that will be present in the responses (see more options by clicking here) and the number of tweets returned in the search.

After the previous definitions, we can perform the request for that we will use the commands below.

response = requests.get(search_url, auth=set_autenticacao, params=query_params)

if response.status_code != 200:
    raise Exception(response.status_code, response.text)

r = response.json()
print(json.dumps(r, indent=4, sort_keys=True))