Skip to main content

Quick Start

Get NoETL running quickly with either PyPI installation or a full local development environment.

Option 1: PyPI Installation (Simplest)​

Install NoETL Python Package​

# Install NoETL
pip install noetl

# Verify installation
python -c "import noetl; print(noetl.__version__)"

Install NoETL CLI (Rust Binary)​

The noetl CLI is a fast, native Rust binary for managing NoETL servers, workers, and playbooks:

# Install with CLI support (recommended)
pip install "noetl[cli]"

# Or install CLI standalone
pip install noetl-cli

# Verify CLI installation
noetl --version

CLI Capabilities:

  • noetl server start/stop - Manage NoETL server
  • noetl worker start/stop - Manage workers
  • noetl db init/validate - Database management
  • noetl build - Build Docker images
  • noetl k8s deploy/redeploy/reset - Kubernetes deployment
  • noetl register playbook - Register playbooks
  • noetl run playbook - Execute playbooks

Option 2: Local Development Environment​

For a complete environment with server, workers, PostgreSQL, and monitoring:

# Clone repository
git clone https://github.com/noetl/noetl.git
cd noetl

# Bootstrap: Install tools and provision complete environment
make bootstrap

What bootstrap does:

  1. Installs required tools: Docker, kubectl, helm, kind, task, psql, pyenv, uv, Rust/Cargo
  2. Creates Kind Kubernetes cluster
  3. Builds NoETL Docker image
  4. Deploys PostgreSQL database
  5. Deploys observability stack (ClickHouse, Qdrant, NATS JetStream)
  6. Deploys monitoring stack (VictoriaMetrics, Grafana, VictoriaLogs)
  7. Deploys NoETL server and workers

Services available after bootstrap:

ServiceURLCredentials
NoETL Serverhttp://localhost:8082-
Grafanahttp://localhost:3000See task grafana
VictoriaMetricshttp://localhost:9428-
PostgreSQLlocalhost:54321demo/demo
ClickHouse HTTPhttp://localhost:30123-
Qdrant HTTPhttp://localhost:30633-
NATS Monitoringhttp://localhost:30822-

Your First Playbook​

1. Create a playbook file​

Create hello_world.yaml:

apiVersion: noetl.io/v2
kind: Playbook
metadata:
  name: hello_world
  path: examples/hello_world
workload:
  message: "Hello from NoETL!"
workflow:
  - step: start
    next:
      - step: greet

  - step: greet
    tool:
      kind: python
      libs: {}
      args:
        message: "{{ workload.message }}"
      code: |
        result = {"status": "success", "data": {"greeting": message}}
    next:
      - step: end

  - step: end

2. Register the playbook​

noetl register playbook hello_world.yaml --host localhost --port 8082

3. Execute the playbook​

noetl run playbook "examples/hello_world" --host localhost --port 8082

4. Check the result​

# List executions
curl http://localhost:8082/api/executions | jq

# Get execution details
curl http://localhost:8082/api/executions/1 | jq

Development Workflow​

After bootstrap, use these common commands:

# Quick development cycle (rebuild + reload)
task dev

# Deploy all components
task deploy-all

# Check cluster health
task test-cluster-health

# View available tasks
task --list

Register Test Fixtures​

NoETL includes example playbooks in tests/fixtures/playbooks/. The CLI supports recursive directory scanning:

# Register all playbooks from a directory (recursive)
noetl register playbook tests/fixtures/playbooks/ --host localhost --port 8082

# Register all credentials from a directory (recursive)
noetl register credential tests/fixtures/credentials/ --host localhost --port 8082

# Register a single playbook
noetl register playbook tests/fixtures/playbooks/hello_world/hello_world.yaml --host localhost --port 8082

# Register a single credential
noetl register credential tests/fixtures/credentials/pg_k8s.yaml --host localhost --port 8082

Cleanup​

# Destroy environment and clean all resources
make destroy

Next Steps​