Data Engineering with AI

Modalità
Online
Lingua
en
Livello
practitioner

Il corso

Ship data pipelines with AI agents doing the first draft: dbt models, Dagster DAGs, quality checks, debugging, and warehouse cost. 7 chapters, about 1.9 hours, for data engineers.

Identità del corso

Materie

ai data engineering course, data engineering with ai agents, dbt ai agent modeling, dagster ai pipelines, data quality as code, ai pipeline debugging, warehouse cost optimization ai, sql and dbt with ai agents, agent assisted data pipelines, learn dbt and dagster

Livello

practitioner

Lingua

en

Programma e obiettivi

Obiettivi
  • Drive an agent to build dbt models against a real schema and style guide instead of a guess
  • Turn a written pipeline spec into a working Dagster DAG with retries and idempotency built in
  • Write data quality checks that actually block a bad load instead of noisy first pass warnings
  • Debug a broken pipeline by driving an agent through Dagster's own run history and check results
  • Generate column docs and lineage notes with an agent, then review them like any other draft
  • Triage a warehouse's query history with an agent and prove one fix with real before and after numbers
Programma
  • Url: https://aiacademy.anthropos.work/chapters/data-engineering-ai-intro/ · Data Engineering with AI: Start Here · Position: 1 · Boot the Anchorwell data lab, a DuckDB warehouse with dbt and Dagster already running, and see how the path's six chapters connect before you touch your own stack.
  • Url: https://aiacademy.anthropos.work/chapters/dataeng-sql-dbt-modeling/ · Agent-Assisted SQL & dbt Modeling · Position: 2 · The dbt workflow where an AI agent drafts and you decide what actually ships to production.
  • Url: https://aiacademy.anthropos.work/chapters/dataeng-pipelines-by-spec/ · Pipelines by Spec: DAGs with Agents · Position: 3 · Write the pipeline spec, drive an agent to build the DAG, and harden it with the retries and idempotency it won't add on its own.
  • Url: https://aiacademy.anthropos.work/chapters/dataeng-data-quality-as-code/ · Data Quality as Code · Position: 4 · Turn an agent's noisy first-pass checks into the few that actually stop a bad load.
  • Url: https://aiacademy.anthropos.work/chapters/dataeng-debugging-pipelines/ · Debugging Broken Pipelines · Position: 5 · Drive an agent through Dagster's run history and check results to find, verify, and fix what actually broke.
  • Url: https://aiacademy.anthropos.work/chapters/dataeng-docs-lineage-definitions/ · Docs, Lineage & Definitions · Position: 6 · Auto-generate column docs and lineage notes with an agent, review them like any other agent draft, then write down one metric definition everyone can point to.
  • Url: https://aiacademy.anthropos.work/chapters/dataeng-warehouse-cost-performance/ · Warehouse Cost & Performance · Position: 7 · Triage a warehouse's query history with an agent, then prove one fix with real before and after numbers.
Competenze acquisite
  • Drive an agent to build dbt models against a real schema and style guide instead of a guess
  • Turn a written pipeline spec into a working Dagster DAG with retries and idempotency built in
  • Write data quality checks that actually block a bad load instead of noisy first pass warnings
  • Debug a broken pipeline by driving an agent through Dagster's own run history and check results
  • Generate column docs and lineage notes with an agent, then review them like any other draft
  • Triage a warehouse's query history with an agent and prove one fix with real before and after numbers

Edizioni

Edizioni

Course Mode: online · Course Workload: PT112M · Mode: online

Corsi simili