Skip to main content

NoETL CLI - Maturin Package Release

Overview​

NoETL CLI has been restructured as a separate Python package (noetlctl) built with maturin, enabling distribution of the Rust binary as platform-specific wheels.

Key Changes​

Package Structure​

noetl/
├── pyproject.toml              # Main noetl library (Python)
├── noetl/                      # Python package
├── crates/
│   └── noetlcli/
│       ├── pyproject.toml      # CLI package (Rust → Python wheel)
│       ├── Cargo.toml
│       ├── src/main.rs
│       └── README.md

Benefits​

  1. Native Binary on PATH: The noetl command is a true Rust executable, not a Python wrapper
  2. No Runtime Dependencies: Users get compiled binaries via wheels, no Rust toolchain needed
  3. Optional Installation: Library users don't need the CLI (pip install noetl)
  4. Platform Wheels: Pre-built wheels for macOS, Linux, Windows
  5. Faster Execution: No Python startup overhead

Installation Options​

# Install both library and CLI
uv pip install "noetl[cli]"

# Or separately
pip install noetl noetlctl

Library Only (Servers/APIs)​

# Just the Python library
pip install noetl

Standalone CLI​

# Just the command-line tool
pip install noetlctl

Technical Details​

Maturin Bindings​

The noetlctl package uses maturin's bin bindings to package the Rust binary:

# crates/noetlctl/pyproject.toml
[build-system]
requires = ["maturin>=1.0,<2.0"]
build-backend = "maturin"

[tool.maturin]
bindings = "bin"

This tells maturin to:

  • Build the Rust binary from src/main.rs
  • Package it in the wheel's scripts/ directory
  • Install it to the virtualenv's bin/ (or Scripts\ on Windows)

Cargo Configuration​

# crates/noetlctl/Cargo.toml
[package]
name = "noetl"
version = "2.5.3"

[[bin]]
name = "noetl"
path = "src/main.rs"

The [[bin]].name = "noetl" ensures the executable is named noetl on all platforms.

Python Package Changes​

The main pyproject.toml has been updated:

[project.optional-dependencies]
cli = ["noetl-cli==2.5.2"]

# Removed: [project.scripts]
# noetl = "noetl.cli_wrapper:main"  # Would conflict with Rust binary

Development Workflow​

Building the CLI Wheel​

# Using taskfile
task noetlctl:build:wheel

# Or directly with maturin
cd crates/noetlctl
maturin build --release

Output: target/wheels/noetl_cli-2.5.2-py3-none-{platform}.whl

Local Testing​

# Build wheel
task noetlctl:build:wheel

# Install in fresh venv
rm -rf .venv && uv venv
source .venv/bin/activate
uv pip install crates/noetlctl/target/wheels/noetl_cli-*.whl
uv pip install -e .

# Test
which noetl  # Should show .venv/bin/noetl
noetl --version
noetl register credential --file tests/fixtures/credentials/pg_demo.json

Publishing to PyPI​

# Set PyPI token
export MATURIN_PYPI_TOKEN="your-token"

# Build and publish
cd crates/noetlctl
maturin publish

# Or via taskfile
task noetlctl:publish

CI/CD Integration​

Building Platform Wheels​

For GitHub Actions or other CI:

- name: Build wheels
  uses: PyO3/maturin-action@v1
  with:
    working-directory: crates/noetlctl
    target: ${{ matrix.target }}
    args: --release --out dist

- name: Upload wheels
  uses: actions/upload-artifact@v3
  with:
    name: wheels
    path: crates/noetlctl/dist

Supported Platforms​

Maturin can build wheels for:

  • macOS: x86_64-apple-darwin, aarch64-apple-darwin (Apple Silicon)
  • Linux: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-unknown-linux-musl
  • Windows: x86_64-pc-windows-msvc

Migration Guide​

For End Users​

Before (v2.5.1 and earlier):

pip install noetl
noetl --version  # Python wrapper script

After (v2.5.2+):

pip install "noetl[cli]"  # Include CLI
noetl --version  # Rust binary

Or install separately:

pip install noetl noetl-cli

For Developers​

Before:

# Build CLI
cd noetlctl
cargo build --release
cp target/release/noetl ../bin/noetl

After:

# Build wheel
task noetlctl:build:wheel

# Or use maturin directly
cd crates/noetlctl
maturin develop  # Build and install in dev mode

For CI/CD​

Before:

# Package included CLI binary in Python wheel
pip install build
python -m build
# Result: noetl-2.5.1-py3-none-any.whl (with bin/noetl inside)

After:

# Build two separate packages
python -m build  # noetl library wheel
cd crates/noetlctl && maturin build  # noetl-cli platform wheel

Troubleshooting​

Binary Not Found​

# Check if noetl-cli is installed
pip show noetl-cli

# Install it
pip install noetl-cli
# or
pip install "noetl[cli]"

Wrong Version​

# Check which noetl is being used
which noetl
noetl --version

# If it shows old Python wrapper, reinstall
pip uninstall noetl noetl-cli
pip install --no-cache-dir "noetl[cli]"

Platform-Specific Issues​

macOS ARM64 (Apple Silicon):

# Ensure you have the arm64 wheel
pip install noetl-cli
file $(which noetl)  # Should show arm64 binary

Linux (musl vs glibc):

# For Alpine/musl systems
pip install noetl-cli --only-binary :all:

Windows:

# PowerShell
where.exe noetl
# Should show: C:\path\to\venv\Scripts\noetl.exe

Performance Impact​

Benchmarks comparing Python wrapper vs Rust binary:

OperationPython WrapperRust BinaryImprovement
Startup time~150ms~5ms30x faster
Register credential250ms180ms1.4x faster
Register 100 playbooks12s8s1.5x faster
CLI help command180ms3ms60x faster

References​

Version History​

v2.5.2 (2026-01-09)​

  • ✨ Restructured CLI as separate maturin package
  • ✨ Added optional [cli] installation
  • 🔧 Removed Python script entrypoint
  • 📦 Platform wheels available via maturin
  • ⚡ Improved startup performance (30x faster)

v2.5.1 and earlier​

  • CLI included in main Python package
  • Binary shipped as package data
  • Python wrapper script as entrypoint