Building a Weather Data Collection System using OpenWeather API, Python, and AWS
Introduction
This article walks you through collecting real-time weather data from the OpenWeatherMap API using Python. The data is stored in AWS S3.
The architectural diagram for the project is below
Prerequisites
Python 3.x - ensure that Python3 is installed on your local machine
AWS CLI to configure AWS credentials on your local machine
OpenWeather API Key - create an account in the OpenWeather website and generate your API key
AWS Account for S3
Project Setup
Create a weather directory for the project and change the directory into it.
mkdir weather-dashboard
cd weather-dashboard
Create the necessary directories and files
This can be done from the terminal or created directly in the Visual Studio Code or any other desired IDE
Install Dependencies
Add the necessary dependencies for the Python app to run successfully
Install the requirement
pip install -r requirements.txt
After I ran this command, I realized that I was getting a particular error on my terminal, see below:
Then, I realized that I needed to create a virtual environment to use the pip command for:
Dependency isolation
Consistency across environments etc.
Hence, I installed the virtual environment using the below:
python3 -m venv .venv -- Linux/MacOs
py -m venv .venv -- Windows
This creates a virtual environment and it is activated as shown below:
source .venv/bin/activate -- Linux/MacOS
source .venv/Scripts/activate -- Windows
Once the virtual environment is activated, you will notice a change in the terminal. Take note that I used WSL2 on my Windows machine.
Environment variable (.env)
The .env file is important for the successful execution of the project because it contains the API keys and the S3 bucket name. The bucket name can be created in the AWS S3 dashboard or can be randomly generated with the command below and added to the .env file
echo "AWS_BUCKET_NAME=weather-dashboard-${RANDOM}" >> .env
Ensure both the bucket name and open weather variable name configured in the .env file match what is inside the Python app
OPENWEATHER_API_KEY=your_openweather_api_key
AWS_BUCKET_NAME=weather-dashboard-30374
AWS Configuration
Once the AWS CLI is installed, create an IAM account on AWS and add the Access key ID, and the Secret access key. Use the command below
aws configure
Once this is completed, you can check your configuration with:
aws configure list
Running the Application
The application is run by going to the src/weather_dashboard.py file and the command is issued:
python3 src/weather_dashboard
I ran the application twice, first for the states in the US and then for Nigeria.
The Python app is available in my GitHub account - https://github.com/gbejula/30DaysDevOpsChallenge
Thank you for following through to the end. Please follow, like, comment, and give feedback.