Flights App with MariaDB Connector/Python
This page is part of MariaDB's SkySQL Documentation.
The parent of this page is: Sample Code
Topics on this page:
Overview
Flights is a web application for analyzing flight information. The application is compatible with MariaDB Connector/Python and the Analytics topologies on MariaDB SkySQL.
Flights demonstrates the power of the Single Node Analytics topology by reading 180 million flight records in real time using lightning fast queries without indexes.
Flights has three components:
React.js front-end web application
Python-based back-end application service
Single Node Analytics back-end database service running on MariaDB SkySQL
Prerequisites
Clone the Repository
Flights is available from a MariaDB Corporation repository on GitHub. The repository contains the SQL code to initialize the database, the API code in Python and other languages, and the React.js front-end web application that connects to the API.
To download the Flights application, clone the repository from GitHub:
$ git clone https://github.com/mariadb-corporation/dev-example-flights.git
$ cd dev-example-flights
This creates a dev-example-flights
directory on your file system.
Provision Database Service
Flights requires a database back-end service using the MariaDB Enterprise ColumnStore storage engine. It is compatible with MariaDB SkySQL Analytics topologies. For the purposes of demonstration, this guide uses the Single Node Analytics topology on MariaDB SkySQL.
For additional information, see "Launch".
Load Flights Data
The repository provides scripts for retrieving data and configuring the database service:
gen_flights_data.sh
retrieves data from the US Department of Transportation. The data is saved in thedata
directory as a series of CSV files with the necessary flight information.The script downloads three decades of flight data, which requires approximately 30G of disk space. It can take a long time to download onto your system and a long time to upload to MariaDB SkySQL, depending on your internet speeds.
create_and_load.sh
creates a set of MariaDB ColumnStore tables on the database service, then imports each of the CSV files into the database.The script is configured to connect to MariaDB SkySQL and expects to find the
skysql_chain.pem
Certificate Authority (CA) file in the same directory.
To load data into the Single Node Analytics database service:
Download flight data:
$ ./get_flight_data.sh
The script downloads the flight data from the US Department of Transportation as a series of CSV files in the
data
directory.Copy the
skysql_chain.pem
file into thedev-example-flights
directory:$ cp ~/Downloads/skysql_chain.pem .
Use the
create_and_load.sh
with your credentials to initialize the database on your SkySQL database service:$ ./create_and_load.sh example.skysql.net 5001 db_user 'db_passwd'
Note that default passwords set for MariaDB SkySQL may require single quotes to prevent the shell from misreading special characters.
Install Back-end Application Service
Flight connects to MariaDB SkySQL using a back-end application service. The repository contains the different versions of the application service, enabling you to test and experiment with different languages and different MariaDB Connectors. These instructions are for Python using MariaDB Connector/Python.
Navigate to the correct api
directory:
$ cd /path/to/dev-example-flights/api/python
Configure the Back-end Application Service
To connect the back-end app to your MariaDB SkySQL database service, you must first configure it to use the appropriate credentials.
Create a
.env
file in theapi/python
directory, set the variables to the login credentials for your MariaDB SkySQL service:DB_HOST_1 = example.skysql.net DB_USER_1 = db_user DB_PASS_1 = db_user_passwd DB_PORT_1 = 5001 DB_NAME_1 = orders
Initialize virtual environment:
$ python3 -m venv .
Activate the virtual environment:
$ sh venv/bin/activate activate
Install dependencies in the virtual environment:
$ ./venv/pip3 install python-dotenv flask mariadb
Start the Back-end Application Service
Once you have the back-end application service configured and installed, you can start the service:
$ python3 api.py
The back-end application service is now running. Install and start the React.js web application to use Flights.
Install Front-end
Flights uses a React.js front-end web application. The front-end connects to the back-end application service and provides a web interface to the user.
To install and start the front-end:
Navigate to the
client
directory:$ cd /path/to/dev-example-flights/client/
Install the web application:
$ npm install
Start the web application:
$ npm start
The React.js web application is now running. You can access it through your web browser at http://localhost:3000.
Live Demonstration
Flights provides a web interface for analyzing flight data. The data comes from United States Department of Transportation and contains 180 million records of flights that were made over the course of thirty years.
Using the interface, enter the origin and destination airports for the flight, then the airline and the range of dates that interest you. Flights queries the data and generates a table listing flight information, and charts noting delays and average minutes delayed.
Flights queries this data from the Single Node Analytics database service, which uses MariaDB Enterprise ColumnStore tables. The columnar table format used by the ColumnStore storage engine enables the application to quickly retrieve the selected rows without using indexes.