How To Build Monitoring Tools with Python

by

For this tutorial we’ll create our own HTTP/HTTPS monitoring based on Python requests. Out there so there are so many free or paid monitoring tools, which is too expensive or too bloat for my use case, which only to monitor the HTTP response and show the messages if there’s any error.

To make it easier to deploy anywhere in these cloud days, we’ll use YAML file as the sources of the domains that we’ll monitor. Let’s create a YAML file called domain.yaml

production:
  - name: wordpress.com
    url: https://wordpress.com
  - name: drupal.org
    url: https://drupal.org
  - name: joomla.org
    url: https://www.joomla.org
staging:
  - name: cloudflare.com
    url: https://www.cloudflare.com
development:
  - name: wordpress.org
    url: https://wordpress.org
  - name: libera.chat
    url: https://libera.chat

the file structure is important, because we’ll coded a script that understand those values.

Python Code

This is the complete scripts, copy the whole line and save it as monitoring.py

#!/usr/bin/env python
 
import requests
import yaml
import os
from datetime import datetime
 
domain_file = os.getenv('DOMAIN_FILE')
 
if not domain_file:
    print("DOMAIN_FILE missing")
    print("exit ...")
    exit()
 
def check_url(url):
  status_code = ""
  error_messages= ""
 
  try:
    r = requests.get(url, timeout=10)
    if(r.ok):
      status_code = r.status_code
    else:
      status_code = r.status_code
  except requests.exceptions.RequestException as err:
    error_messages = err
  finally:
    if status_code is not None:
        http_status_code = status_code
    else:
        http_status_code = ""
    if error_messages is not None:
        err_msg = error_messages
    else:
        err_msg = ""
 
    response = dict();
    response['status_code'] = http_status_code
    response['err_msg'] = str(err_msg).replace("'",'')
    return response
 
with open(domain_file, 'r') as file:
  urls = yaml.safe_load(file).items()
 
  for key,values in urls:
    for value in values:
      url_response = check_url(value['url'])
      dt = datetime.now()
 
      format_msg = {
      "time": dt.isoformat(),
      "env": key,
      "name": value['name'],
      "url": value['url'],
      "status_code": url_response['status_code'],
      "messages": url_response['err_msg']
      }
 
      print(format_msg)

Run The Script

To run the Python script, follow below instructions

# full path to domain.yaml
export DOMAIN_FILE=/home/jack/scripts/domain.yaml
# execute the python script
# python /FULL/PATH/TO/monitoring.py
python /home/jack/monitoring.py

output from command above

{'time': '2025-01-04T21:28:10.593535', 'env': 'production', 'name': 'wordpress.com', 'url': 'https://wordpress.com', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T21:28:11.547092', 'env': 'production', 'name': 'drupal.org', 'url': 'https://drupal.org', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T21:28:12.524504', 'env': 'production', 'name': 'joomla.org', 'url': 'https://www.joomla.org', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T21:28:12.756726', 'env': 'staging', 'name': 'cloudflare.com', 'url': 'https://www.cloudflare.com', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T21:28:13.794606', 'env': 'development', 'name': 'wordpress.org', 'url': 'https://wordpress.org', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T21:28:13.926251', 'env': 'development', 'name': 'libera.chat', 'url': 'https://libera.chat', 'status_code': 200, 'messages': ''}

Build Docker Image

To make it easier to deploy as docker images or on Kubernetes, let’s build the docker image from that script. Create a new Dockerfile with following codes

FROM python:3.13-slim-bookworm
ENV DOMAIN_FILE=/app/domain.yaml
ENV PYTHONPATH=/app
USER root
RUN groupadd -g 10000 app && useradd -u 10000 -g app -m -d /app app
USER app
WORKDIR /app
RUN ls -lah /app
COPY --chown=app:app domain.yaml monitoring.py /app/
RUN pip install --no-cache-dir requests pyyaml
CMD ["python", "monitoring.py"]

then build the image.

docker build . -t atetux/python-monitoring

this image only available in your local, before it can deployed to the Kubernetes you need to deploy it to docker registry first, which is not covered by this tutorial.

Run the docker image

$ docker run atetux/python-monitoring
# output
{'time': '2025-01-04T15:11:22.944968', 'env': 'production', 'name': 'wordpress.com', 'url': 'https://wordpress.com', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T15:11:24.026038', 'env': 'production', 'name': 'drupal.org', 'url': 'https://drupal.org', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T15:11:25.025014', 'env': 'production', 'name': 'joomla.org', 'url': 'https://www.joomla.org', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T15:11:25.185212', 'env': 'staging', 'name': 'cloudflare.com', 'url': 'https://www.cloudflare.com', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T15:11:26.240853', 'env': 'development', 'name': 'wordpress.org', 'url': 'https://wordpress.org', 'status_code': 200, 'messages': ''}
{'time': '2025-01-04T15:11:26.476771', 'env': 'development', 'name': 'libera.chat', 'url': 'https://libera.chat', 'status_code': 200, 'messages': ''}

python http monitoring tools

Leave a Comment