Quick Start

Go from zero to a generated Airflow DAG in under 5 minutes.

Prerequisites

Python ≥ 3.13
uv — package manager and script runner
ruff — linter and formatter (used for post-processing generated DAGs)

Installation

Install uv

bash

# Linux / macOS
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Install ruff

bash

# Linux / macOS
curl -LsSf https://astral.sh/ruff/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/ruff/install.ps1 | iex"

Install project dependencies

bash

# Install runtime + dev toolchain
uv sync --group dev

Your First DAG in 60 Seconds

Step 1: Create a YAML spec

Create a file called my_pipeline.yaml:

yaml

metadata:
  title: "My First Pipeline"
  owner: "data-team@example.com"
  email: "data-team@example.com"
  version: "1.0.0"
  jira: "PROJ-001"
  developer_name: "my_pipeline"

dag:
  dag_id: "my_first_dag"
  description: "A simple BigQuery pipeline."
  schedule: "@daily"
  start_date: "2026-01-01"
  dagrun_timeout: 3600
  catchup: false
  tags:
    - "team:data"

gcp:
  project_id: "my-gcp-project"
  location: "US"

default_args:
  retries: 2
  retry_delay: 60
  email:
    - "data-team@example.com"

tasks:
  - task_id: "start"
    operator: EmptyOperator

  - task_id: "load_data"
    operator: BigQueryInsertJobOperator
    sql: "sql/load_data.sql"

  - task_id: "end"
    operator: EmptyOperator

dependencies:
  - "start >> load_data >> end"

Step 2: Generate the DAG

bash

dagsmith generate my_pipeline.yaml

DagSmith validates the YAML, generates a Python DAG file, and formats it with ruff. The output lands in dags/my_first_dag.py.

Step 3: Validate without generating

bash

# Quick validation
dagsmith validate my_pipeline.yaml

# Strict mode (CI gate)
dagsmith validate my_pipeline.yaml --strict

Common Commands

bash

# Generate all DAGs from a directory
dagsmith generate examples/

# Generate a single DAG
dagsmith generate examples/01_simple_bq_pipeline.yaml

# Generate with explicit output directory
dagsmith generate examples/ -o dags/

# Dry run (validate + render, no file writes)
dagsmith generate examples/ --dry-run

# List all registered operators and sensors
dagsmith list

# List only BigQuery operators
dagsmith list --origin third_party --type operator

# Resolve variables and preview expanded YAML
dagsmith resolve examples/01_simple_bq_pipeline.yaml

Development Toolchain

Tool	Purpose	Command
`uv`	Package manager + script runner	`uv sync --group dev`
`ruff`	Lint + format (line-length: 110)	`uv run ruff check --fix . && uv run ruff format .`
`mypy`	Strict type checking	`uv run mypy .`
`pytest`	Testing (coverage ≥ 80%)	`uv run pytest`
`yamllint`	YAML linting	`yamllint -c yamllint-config.yml .`

← Previous

Overview

CLI Reference