tdp-lib

TDP Lib

tdp-lib is a Python library built on top of Ansible. It is designed for managing TDP clusters, offering advanced tools to extend Ansible’s capabilities. Key features include:

Creating a Directed Acyclic Graph (DAG) of tasks to manage dependencies and relationships between services and components.
Centralized configuration management through a unified variable definition system.

tdp-lib can be utilized in various ways: as a Python library, through an admin CLI, via a REST API (see tdp-server), or through a web interface (see tdp-ui).

Pre-requisites

To use tdp-lib, ensure you have the following prerequisites:

Python 3.9 or higher. Higher version are not guarantee to work, only 3.9 is tested.
A relational database management system (RDBMS), such as PostgreSQL or SQLite.

Optional dependencies for DAG visualization:

Graphviz for graphical representation of DAGs.

Installation

Set the following environment variables:

TDP_COLLECTION_PATH: Specifies the file path(s) to the necessary collection(s). tdp-collection is mandatory. Multiple collections can be specified, separated by a colon : (e.g., tdp-collection-extras, tdp-observability).
TDP_RUN_DIRECTORY: Path to the working directory of the TDP deployment (where ansible.cfg, inventory.ini, and topology.ini are located).
TDP_DATABASE_DSN: Database DSN (Data Source Name) for the chosen RDBMS.
TDP_VARS: Path to the folder containing configuration variables.

Ensure Ansible is configured to use the tosit.tdp.inventory plugin. Example ansible.cfg:

[defaults]
inventory=your_inventory,..,~/tdp_vars

[inventory]
enable_plugins = tosit.tdp.inventory,..,your_plugins

Install the library:

# Create and activate a virtual environment
python -m venv .venv
source .venv/bin/activate
# Install the dependencies
pip install "tdp-lib[visualization]@https://github.com/TOSIT-IO/tdp-lib/tarball/1.0.0"
# Initialize the database and tdp_vars
tdp init

Supported databases

tdp-lib uses SQLite by default. Other backends are supported:

PostgreSQL - install with pip install tdp-lib[postgresql]
MySQL/MariaDB - install with pip install tdp-lib[mysql]

CLI Usage

[!NOTE] This section is a work in progress.

tdp --help

Contributing

Contributions are welcome! Here are some guidelines specific to this project:

Dev environment

Use uv for development:

# Install dependencies
uv sync --all-extras

Testing

Tests are based on Pytest:

uv run pytest

tdp-lib is supporting SQLite, PostgreSQL and MariaDB. A compose.yaml is provided to run test accross all suported environments:

docker compose -f dev/docker-compose.yaml up -d
uv run pytest tests --database-dsn 'postgresql+psycopg2://postgres:postgres@localhost:5432/tdp' --database-dsn 'mysql+pymysql://mysql:mysql@localhost:3306/tdp' --database-dsn 'mysql+pymysql://mariadb:mariadb@localhost:3307/tdp'
docker compose -f dev/docker-compose.yaml down -v

Contributions

Commit messages must adhere to the Conventional Commits standard.

Code must be formated and ensure the linting rules before being merged. This project use Ruff:

# Format the code and reorder imports
uv run ruff check --select I --fix && uv run ruff format
# Check the code for linting issues
uv run ruff check
# Lint the code
uv run ruff check --fix

This site is open source. Improve this page.