AWS Glue Triggers

AWS Glue Triggers

• Initiates an ETL job.
• Can be defined based on a scheduled time or an event.

Types of Glue Triggers:

1. Scheduled Triggers:
   • Cron Expressions: Run Crawlers or Jobs on a recurring schedule using Cron or simplified rate expression (e.g., run every hour, every Monday at 8 am, etc.).
   • Benefits: Regularly update your Data Catalog with fresh data or run ETL jobs on a predictable timetable.
2. On-demand Triggers:
   • Manual Execution: Start a Glue Crawler or ETL Job manually through the console or API.
   • Benefits: Useful for one-time execution, testing, or backfilling data.
3. Event-based Triggers:
   • Conditional Start: Launch a Crawler or Job in response to specific events in other AWS services.
     Examples:
     • An S3 object is created or modified in a specific bucket.
     • An AWS Lambda Function execution succeeds.
     • A previous Job or Crawler completes its run.
   • Benefits: Create dynamic, reactive data processing pipelines.

Hands-on Lab