12.12.2020 14:34

Creating Web APIs with Python and Flask

For this tutorial, you will need Python 3 and the Flask web framework. You’ll also require a web browser (such as Firefox) and a text editor (such as Notepad++ or BBEdit).

What is an API?

If you’ve heard the term API before, chances are it’s been used not to refer to APIs in general, but instead to a specific kind of API, the web API. A web API allows for information or functionality to be manipulated by other programs via the internet. For example, with Twitter’s web API, you can write a program in a language like Python or Javascript that can perform tasks such as favoriting tweets or collecting tweet metadata.

In programming more generally, the term API, short for Application Programming Interface, refers to a part of a computer program designed to be used or manipulated by another program, as opposed to an interface designed to be used or manipulated by a human.

Computer programs frequently need to communicate amongst themselves or with the underlying operating system, and APIs are one way they do it. In this tutorial, however, we’ll be using the term API to refer specifically to web APIs.

When to Create an API

In general, consider an API if:

Your data set is large, making download via FTP unwieldy or resource-intensive.

Your users will need to access your data in real time, such as for display on another website or as part of an application.

Your data changes or is updated frequently.

Your users only need access to a part of the data at any one time.

Your users will need to perform actions other than retrieve data, such as contributing, updating, or deleting data.

If you have data you wish to share with the world, an API is one way you can get it into the hands of others. However, APIs are not always the best way of sharing data with users. If the size of the data you are providing is relatively small, you can instead provide a “data dump” in the form of a downloadable JSON, XML, CSV, or SQLite file. Depending on your resources, this approach can be viable up to a download size of a few gigabytes.

Remember that you can provide both a data dump and an API, and individual users may find one or the other to better match their use case. Open Library, for example, provides both a data dump and an API, each of which serves different use cases for different users. For more info Learn Python Online

API Terminology

When using or building APIs, you will encounter these terms frequently:

HTTP (Hypertext Transfer Protocol) is the primary means of communicating data on the web. HTTP implements a number of “methods,” which tell which direction data is moving and what should happen to it. The two most common are GET, which pulls data from a server, and POST, which pushes new data to a server.

URL (Uniform Resource Locator) - An address for a resource on the web, such as https://programminghistorian.org/about. A URL consists of a protocol (http://), domain (programminghistorian.org), and optional path (/about). A URL describes the location of a specific resource, such as a web page. When reading about APIs, you may see the terms URL, request, URI, or endpoint used to describe adjacent ideas. This tutorial will prefer the terms URL and request to avoid complication. You can follow a URL or make a GET request in your browser, so you won’t need any special software to make requests in this tutorial.

JSON (JavaScript Object Notation) is a text-based data storage format that is designed to be easy to read for both humans and machines. JSON is generally the most common format for returning data through an API, XML being the second most common.

REST (REpresentational State Transfer) is a philosophy that describes some best practices for implementing APIs. APIs designed with some or all of these principles in mind are called REST APIs. While the API outlined in this lesson uses some REST principles, there is a great deal of disagreement around this term. For this reason, I do not describe the example APIs here as REST APIs, but instead as web or HTTP APIs. Get advanced skills from Python Online Classes

Using APIs

Why Use APIs as a Researcher?

The primary focus of this lesson is on creating an API, not exploring or using an API that has already been implemented. However, before we start building our own API, it may be useful to discuss how APIs are useful for researchers. In this section, we’ll see how APIs can be useful for approaching historical, textual, or sociological questions using a “macroscopic” or “distant reading” approach that makes use of relatively large amounts of information. In doing so, we’ll familiarize ourselves with the basic elements of a good API. Considering APIs from the perspective of a user will come in useful when we begin to design our own API later in the lesson.

Implementing Our API

Overview

This section will show you how to build a prototype API using Python and the Flask web framework. Our example API will take the form of a distant reading archive—a book catalog that goes beyond standard bibliographic information to include data of interest to those working on digital projects. In this case, besides title and date of publication, our API will also serve the first sentence of each book. This should be enough data to allow us to envision some potential research questions without overwhelming us as we focus on the design of our API.

We’ll begin by using Flask to create a home page for our site. In this step, we’ll learn the basics of how Flask works and make sure our software is configured correctly. Once we have a small Flask application working in the form of a home page, we’ll iterate on this site, turning it into a functioning API.

Creating a Basic Flask Application

Flask is a web framework for Python, meaning that it provides functionality for building web applications, including managing HTTP requests and rendering templates. In this section, we will create a basic Flask application. In later sections, we’ll add to this application to create our API. Don’t worry if you don’t understand each individual line of code yet—explanations will be forthcoming once you have this initial version of the application working.

Why Flask?

Python has a number of web frameworks that can be used to create web apps and APIs. The most well-known is Django, a framework that has a set project structure and which includes many built-in tools. This can save time and effort for experienced programmers, but can be overwhelming. Flask applications tend to be written on a blank canvas, so to speak, and so are more suited to a contained application such as our prototype API.

First, create a new folder on your computer that will serve as a project folder. This can be in your Desktop folder, but I recommend creating a dedicated projects folder for this and similar projects. This tutorial will assume that the files related to this lesson will be stored in a folder called api inside a folder named projects in your home directory. If you need help with navigation on the command line. For more practical skills learn from Python Online Training

In macOS, you can directly create a an api folder inside a projects folder in your home directory with this terminal command:

mkdir -p ~/projects/api

On Windows, you can create the api folder with these commands in your cmd command line environment:

md projects cd projects md api

You can also create the projects and api folders using your operating system’s graphical user interface.

Next, open a text editor (such as Notepad++ or BBEdit) and enter the following code:

import flask app = flask.Flask(__name__) app.config["DEBUG"] = True @app.route('/', methods=['GET']) def home(): return "<h1>Distant Reading Archive</h1><p>This site is a prototype API for distant reading of science fiction novels.</p>" app.run()

Save this code as api.py in the api folder you created for this tutorial.

Running the Application

In the command line, navigate to your api folder:

cd projects/api

You can check if you’re in the correct folder by running the pwd command. Once you’re in your project directory, run the Flask application with the command:

python api.py

You should see output similar to this:

* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

You may also see some lines related to debugging. This message means that Flask is running your application locally (on your computer) at that address. Follow the link above, http://127.0.0.1:5000/, using your web browser to see the running application:

Congratulations, you’ve created a working web application!

What Flask Does

Now that we have a homepage for our archive, let’s talk about how Flask works and what the above code is doing.

Flask maps HTTP requests to Python functions. In this case, we’ve mapped one URL path (‘/’) to one function, home. When we connect to the Flask server at http://127.0.0.1:5000/, Flask checks if there is a match between the path provided and a defined function. Since /, or no additional provided path, has been mapped to the home function, Flask runs the code in the function and displays the returned result in the browser. In this case, the returned result is HTML markup for a home page welcoming visitors to the site hosting our API.

The process of mapping URLs to functions is called routing. The

@app.route('/', methods=['GET'])

syntax is the part of the program that lets Flask know that this function, home, should be mapped to the path /. The methods list (methods=['GET']) is a keyword argument that lets Flask know what kind of HTTP requests are allowed. We’ll only be using GET requests in this tutorial, but many web applications need to use both GET requests (to send data from the application to the user) and POST requests (to receive data from a user).

Below are brief explanations of the other components of the application:

import flask — Imports the Flask library, making the code available to the rest of the application.

app = flask.Flask(__name__) — Creates the Flask application object, which contains data about the application and also methods (object functions) that tell the application to do certain actions. The last line, app.run(), is one such method.

app.config["DEBUG"] = True — Starts the debugger. With this line, if your code is malformed, you’ll see an error when you visit your app. Otherwise you’ll only see a generic message such as Bad Gateway in the browser when there’s a problem with your code.

app.run() — A method that runs the application server.

While it’s useful to have a familiarity with what’s going on in the script, don’t worry if you don’t understand precisely what every element is doing at this stage. If you understand the general outline of how this portion works, the details of how Flask renders pages are likely to become more understandable as we continue to develop our API. For more info Python Certification Course

Creating the API

Now that we have a running Flask application and know a little about what Flask does, we’re finally ready to implement a small API with data that we’ll define right in our application.

We’ll be adding our data as a list of Python dictionaries. Dictionaries in Python group pairs of keys and values, like this:

{ 'key': 'value', 'key': 'value' }

The key identifies the type of information represented, such as title or id. The value is the actual data. For example, a short telephone book might take this format:

[ { 'name': 'Alexander Graham Bell', 'number': '1-333-444-5555' }, { 'name': 'Thomas A. Watson', 'number': '1-444-555-6666' } ]

The above phone book is a list of two dictionaries. Each dictionary is a phone book entry consisting of two keys, name and number, each paired with a value that provides the actual information.

Let’s add some data (entries on three science fiction novels) as a list of dictionaries. Each dictionary will contain ID number, title, author, first sentence, and year of publication for each book. Finally, we’ll add a new function: a route that will allow a visitor to access our data.

Replace our previous code in api.py with the code below:

Run the code (navigate to your api folder in the command line and enter python api.py).

You should see JSON output for the three entries in our test catalog. Flask provides us with a jsonify function that allows us to convert lists and dictionaries to JSON format. In the route we created, our book entries are converted from a list of Python dictionaries to JSON before being returned to a user.

At this point, you’ve created a working, if limited, API. In the next section, we’ll allow users to find books via more specific data, such as an entry’s ID.

Finding Specific Resources

Right now, users can only view our entire database—they can’t filter or find specific resources. While this isn’t a problem with our test catalog, this will quickly become less useful as we add data. In this section, we’ll add a function that allows users to filter their returned results using a more specific request.

Below is the code for our new application with filtering capability. As before, we’ll examine the code more carefully once you have it running.

import flask from flask import request, jsonify app = flask.Flask(__name__) app.config["DEBUG"] = True # Create some test data for our catalog in the form of a list of dictionaries. books = [ {'id': 0, 'title': 'A Fire Upon the Deep', 'author': 'Vernor Vinge', 'first_sentence': 'The coldsleep itself was dreamless.', 'year_published': '1992'}, {'id': 1, 'title': 'The Ones Who Walk Away From Omelas', 'author': 'Ursula K. Le Guin', 'first_sentence': 'With a clamor of bells that set the swallows soaring, the Festival of Summer came to the city Omelas, bright-towered by the sea.', 'published': '1973'}, {'id': 2, 'title': 'Dhalgren', 'author': 'Samuel R. Delany', 'first_sentence': 'to wound the autumnal city.', 'published': '1975'} ] @app.route('/', methods=['GET']) def home(): return '''<h1>Distant Reading Archive</h1> <p>A prototype API for distant reading of science fiction novels.</p>''' @app.route('/api/v1/resources/books/all', methods=['GET']) def api_all(): return jsonify(books) @app.route('/api/v1/resources/books', methods=['GET']) def api_id(): # Check if an ID was provided as part of the URL. # If ID is provided, assign it to a variable. # If no ID is provided, display an error in the browser. if 'id' in request.args: id = int(request.args['id']) else: return "Error: No id field provided. Please specify an id." # Create an empty list for our results results = [] # Loop through the data and match results that fit the requested ID. # IDs are unique, but other fields might return many results for book in books: if book['id'] == id: results.append(book) # Use the jsonify function from Flask to convert our list of # Python dictionaries to the JSON format. return jsonify(results) app.run()

Once you’ve updated your API with the api_id function, run your code as before (python api.py from your api directory) and visit the below URLs to test the new filtering capability:

Each of these should return a different entry, except for the last, which should return an empty list: [], since there is no book for which the id value is 3. (Counting in programming typically starts from 0, so id=3 would be a request for a nonexistent fourth item.) In the next section, we’ll explore our updated API in more detail.

Understanding Our Updated API

In this code, we first create a new function, called api_id, with the @app.route syntax that maps the function to the path /api/v1/resources/books. That means that this function will run when we access (Note that accessing this link without providing an ID will give the error message we provided in the code: Error: No id field provided. Please specify an id.)

Inside our function, we do two things:

First, examine the provided URL for an id and select the books that match that id. The id must be provided like this: ?id=0. Data passed through URLs like this (after the ?) are called query parameters—we’ve seen them before when we worked with the Chronicling America API. They’re a feature of HTTP used for filtering for specific kinds of data.

This part of the code determines if there is a query parameter, like ?id=0, and then assigns the provided ID to a variable.

if 'id' in request.args: id = int(request.args['id']) else: return "Error: No id field provided. Please specify an id."

Then this section moves through our test catalog of books, matches those books that have the provided ID, and appends them to the list that will be returned to the user:

for book in books: if book['id'] == id: results.append(book)

Finally, the return jsonify(results) line takes the list of results and renders them in the browser as JSON.

If you’ve gotten this far, you’ve created an actual API. Celebrate! At the end of this lesson, you’ll be exposed to a somewhat more complex API that uses a database, but most of the principles and patterns we’ve used so far will still apply. In the next section, we’ll discuss some guidelines for creating a well-designed API that others will actually want to use. In the last section of the tutorial, we’ll apply these principles to a version of our API that pulls in results from a database.

API Design Principles

Thus far, we’ve created a working API with test data that we’ve provided right in our application. Our next version of our API will pull in data from a database before providing it to a user. It will also take additional query parameters, allowing users to filter by fields other than ID.

Before building more functionality into our application, let’s reflect on some of the API design decisions that we’ve made so far. Two aspects of a good API are usability and maintainability, and as we build more functionality into our API, we’ll be keeping many of the following considerations in mind.

Designing Requests

The prevailing design philosophy of modern APIs is called REST. For our purposes, the most important thing about REST is that it’s based on the four methods defined by the HTTP protocol: POST, GET, PUT, and DELETE. These correspond to the four traditional actions performed on data in a database: CREATE, READ, UPDATE, and DELETE. In this tutorial, we’ll only be concerned with GET requests, which correspond to reading from a database.

Because HTTP requests are so integral to using a REST API, many design principles revolve around how requests should be formatted. We’ve already created one HTTP request, which returns all books provided in our sample data. To understand the considerations that go into formatting this request, let’s first consider a weak or poorly-designed example of an API endpoint:

http://api.example.com/getbook/10

The formatting of this request has a number of issues. The first is semantic—in a REST API, our verbs are typically GET, POST, PUT, or DELETE, and are determined by the request method rather than in the request URL. That means that the word “get” should not appear in our request, since “get” is implied by the fact that we’re using a HTTP GET method. In addition, resource collections such as books or users should be denoted with plural nouns. This makes it clear when an API is referring to a collection (books) or an entry (book). Incorporating these principles, our API would look like this:

http://api.example.com/books/10

The above request uses part of the path (/10) to provide the ID. While this is not an uncommon approach, it’s somewhat inflexible—with URLs constructed in this manner, you can generally only filter by one field at a time. Query parameters allow for filtering by multiple database fields and make more sense when providing “optional” data, such as an output format:

http://api.example.com/books?author=Ursula+K.+Le Guin&published=1969&output=xml

When designing how requests to your API should be structured, it also makes sense to plan for future additions. Even if the current version of your API serves information on only one type of resource—books, for example—it makes sense to plan as if you might add other resources or non-resource functionality to your API in the future:

http://api.example.com/resources/books?id=10

Adding an extra segment on your path such as “resources” or “entries” gives you the option to allow users to search across all resources available, making it easier for you to later support requests such as these:

https://api.example.com/v1/resources/images?id=10 https://api.example.com/v1/resources/all

Another way to plan for your API’s future is to add a version number to the path. This means that, should you have to redesign your API, you can continue to support the old version of the API under the old version number while releasing, for example, a second version (v2) with improved or different functionality. This way, applications and scripts built using the old version of your API won’t cease to function after your upgrade.