Airflow API (Stable)
Overview To facilitate management, Apache Airflow supports a range of REST API endpoints across its objects. This section provides an overview of the API…
Communication#openapi#messaging
Fork copies the collection into your workspace, without the publisher's secrets, so you can add your own key and send requests. Star saves it to your starred APIs (Account › Marketplace) and helps others find it.
Overview
Overview
To facilitate management, Apache Airflow supports a range of REST API endpoints across its
objects.
This section provides an overview of the API design, methods, and supported use cases.
Most of the endpoints accept JSON as input and return JSON responses.
This means that you must usually add the following headers to your request:
Content-type: application/json
Accept: application/jsonResources
The term resource refers to a single type of object in the Airflow metadata. An API is broken up by its
endpoint's corresponding resource.
The name of a resource is typically plural and expressed in camelCase. Example: dagRuns.
Resource names are used as part of endpoint URLs, as well as in API parameters and responses.
CRUD Operations
The platform supports Create, Read, Update, and Delete operations on most resources.
You can review the standards for these operations and their standard parameters below.
Some endpoints have special behavior as exceptions.
Create
To create a resource, you typically submit an HTTP POST request with the resource's required metadata
in the request body.
The response returns a 201 Created response code upon success with the resource's metadata, including
its internal id, in the response body.
Read
The HTTP GET request can be used to read a resource or to list a number of resources.
A resource's id can be submitted in the request parameters to read a specific resource.
The response usually returns a 200 OK response code upon success, with the resource's metadata in
the response body.
If a GET request does not include a specific resource id, it is treated as a list request.
The response usually returns a 200 OK response code upon success, with an object containing a list
of resources' metadata in the response body.
When reading resources, some common query parameters are usually available. e.g.:
v1/connections?limit=25&offset=25| Query Parameter | Type | Description |
|---|---|---|
| limit | integer | Maximum number of objects to fetch. Usually 25 by default |
| offset | integer | Offset after which to start returning objects. For use with limit query parameter. |
Update
Updating a resource requires the resource id, and is typically done using an HTTP PATCH request,
with the fields to modify in the request body.
The response usually returns a 200 OK response code upon success, with information about the modified
resource in the response body.
Delete
Deleting a resource requires the resource id and is typically executing via an HTTP DELETE request.
The response usually returns a 204 No Content response code upon success.
Conventions
- Resource names are plural and expressed in camelCase.
- Names are consistent between URL parameter name and field name.
- Field names are in snake_case.
{
"name": "string",
"slots": 0,
"occupied_slots": 0,
"used_slots": 0,
"queued_slots": 0,
"open_slots": 0
}Update Mask
Update mask is available as a query parameter in patch endpoints. It is used to notify the
API which fields you want to update. Using update_mask makes it easier to update objects
by helping the server know which fields to update in an object instead of updating all fields.
The update request ignores any fields that aren't specified in the field mask, leaving them with
their current values.
Example:
resource = request.get('/resource/my-id').json()
resource['my_field'] = 'new-value'
request.patch('/resource/my-id?update_mask=my_field', data=json.dumps(resource))Versioning and Endpoint Lifecycle
- API versioning is not synchronized to specific releases of the Apache Airflow.
- APIs are designed to be backward compatible.
- Any changes to the API will first go through a deprecation phase.
Trying the API
You can use a third party client, such as curl, HTTPie,
Postman or the Insomnia rest client to test
the Apache Airflow API.
Note that you will need to pass credentials data.
For e.g., here is how to pause a DAG with curl, when basic authorization is used:
curl -X PATCH 'https://example.com/api/v1/dags/{dag_id}?update_mask=is_paused' \
-H 'Content-Type: application/json' \
--user "username:password" \
-d '{
"is_paused": true
}'Using a graphical tool such as Postman or Insomnia,
it is possible to import the API specifications directly:
- Download the API specification by clicking the Download button at top of this document
- Import the JSON specification in the graphical tool of your choice.
- In Postman, you can click the import button at the top
- With Insomnia, you can just drag-and-drop the file on the UI
Note that with Postman, you can also generate code snippets by selecting…
Endpoints 73
Config
- GETGet current configuration
/config
Connection
- GETList connections
/connections?limit=100&offset=0&order_by=string - POSTCreate a connection
/connections - POSTTest a connection
/connections/testTest a connection. *New in version 2.2.0* - GETGet a connection
/connections/:connection_id - DELETEDelete a connection
/connections/:connection_id - PATCHUpdate a connection
/connections/:connection_id?update_mask=string
DAG
- GETGet a source code
/dagSources/:file_tokenGet a source code using file token. - GETList DAGs
/dags?limit=100&offset=0&order_by=string&tags=string&only_active=true&dag_id_pattern=stringList DAGs in the database. dag_id_pattern can be set to match dags of a specific pattern - PATCHUpdate DAGs
/dags?limit=100&offset=0&tags=string&update_mask=string&only_active=true&dag_id_pattern=stringUpdate DAGs of a given dag_id_pattern using UpdateMask. This endpoint allows specifying ~ as the dag_id_pattern to update all DAGs. *New in version 2.3.0* - GETGet basic information about a DAG
/dags/:dag_idPresents only information available in database (DAGModel). If you need detailed information, consider using GET /dags/{dag_id}/details. - DELETEDelete a DAG
/dags/:dag_idDeletes all metadata related to the DAG, including finished DAG Runs and Tasks. Logs are not deleted. This action cannot be undone. *New in version 2.2.0* - PATCHUpdate a DAG
/dags/:dag_id?update_mask=string - POSTClear a set of task instances
/dags/:dag_id/clearTaskInstancesClears a set of task instances associated with the DAG for a specified date range. - GETGet a simplified representation of DAG
/dags/:dag_id/detailsThe response contains many DAG attributes, so the response can be large. If possible, consider using GET /dags/{dag_id}. - GETGet tasks for DAG
/dags/:dag_id/tasks?order_by=string - GETGet simplified representation of a task
/dags/:dag_id/tasks/:task_id - POSTSet a state of task instances
/dags/:dag_id/updateTaskInstancesStateUpdates the state for multiple task instances simultaneously.
DAGRun
- GETList DAG runs
/dags/:dag_id/dagRuns?limit=100&offset=0&execution_date_gte=2024-01-01T00:00:00Z&execution_date_lte=2024-01-01T00:00:00Z&start_date_gte=2024-01-01T00:00:00Z&start_date_lte=2024-01-01T00:00:00Z&end_date_gte=2024-01-01T00:00:00Z&end_date_lte=2024-01-01T00:00:00Z&state=string&order_by=stringThis endpoint allows specifying ~ as the dag_id to retrieve DAG runs for all DAGs. - POSTTrigger a new DAG run
/dags/:dag_id/dagRuns - GETGet a DAG run
/dags/:dag_id/dagRuns/:dag_run_id - DELETEDelete a DAG run
/dags/:dag_id/dagRuns/:dag_run_id - PATCHModify a DAG run
/dags/:dag_id/dagRuns/:dag_run_idModify a DAG run. *New in version 2.2.0* - POSTClear a DAG run
/dags/:dag_id/dagRuns/:dag_run_id/clearClear a DAG run. *New in version 2.4.0* - PATCHUpdate the DagRun note.
/dags/:dag_id/dagRuns/:dag_run_id/setNoteUpdate the manual user note of a DagRun. *New in version 2.5.0* - GETGet dataset events for a DAG run
/dags/:dag_id/dagRuns/:dag_run_id/upstreamDatasetEventsGet datasets for a dag run. *New in version 2.4.0* - POSTList DAG runs (batch)
/dags/~/dagRuns/listThis endpoint is a POST to allow filtering across a large number of DAG IDs, where as a GET it would run in to maximum HTTP request URL length limit.
EventLog
- GETList log entries
/eventLogs?limit=100&offset=0&order_by=stringList log entries from event log. - GETGet a log entry
/eventLogs/:event_log_id
ImportError
- GETList import errors
/importErrors?limit=100&offset=0&order_by=string - GETGet an import error
/importErrors/:import_error_id
Monitoring
- GETGet instance status
/healthGet the status of Airflow's metadatabase and scheduler. It includes info about metadatabase and last heartbeat of scheduler. - GETGet version information
/version
Pool
- GETList pools
/pools?limit=100&offset=0&order_by=string - POSTCreate a pool
/pools - GETGet a pool
/pools/:pool_name - DELETEDelete a pool
/pools/:pool_name - PATCHUpdate a pool
/pools/:pool_name?update_mask=string
Provider
- GETList providers
/providersGet a list of providers. *New in version 2.1.0*
TaskInstance
- GETList task instances
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances?execution_date_gte=2024-01-01T00:00:00Z&execution_date_lte=2024-01-01T00:00:00Z&start_date_gte=2024-01-01T00:00:00Z&start_date_lte=2024-01-01T00:00:00Z&end_date_gte=2024-01-01T00:00:00Z&end_date_lte=2024-01-01T00:00:00Z&duration_gte=0.0&duration_lte=0.0&state=string&pool=string&queue=string&limit=100&offset=0This endpoint allows specifying ~ as the dag_id, dag_run_id to retrieve DAG runs for all DAGs and DAG runs. - GETGet a task instance
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id - PATCHUpdates the state of a task instance
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_idUpdates the state for single task instance. *New in version 2.5.0* - GETList extra links
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/linksList extra links for task instance. - GETList mapped task instances
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/listMapped?limit=100&offset=0&execution_date_gte=2024-01-01T00:00:00Z&execution_date_lte=2024-01-01T00:00:00Z&start_date_gte=2024-01-01T00:00:00Z&start_date_lte=2024-01-01T00:00:00Z&end_date_gte=2024-01-01T00:00:00Z&end_date_lte=2024-01-01T00:00:00Z&duration_gte=0.0&duration_lte=0.0&state=string&pool=string&queue=string&order_by=stringGet details of all mapped task instances. *New in version 2.3.0* - GETGet logs
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/logs/:task_try_number?full_content=true&map_index=0&token=stringGet logs for a specific task instance and its try number. - PATCHUpdate the TaskInstance note.
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/setNoteUpdate the manual user note of a non-mapped Task Instance. *New in version 2.5.0* - GETGet a mapped task instance
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/:map_indexGet details of a mapped task instance. *New in version 2.3.0* - PATCHUpdates the state of a mapped task instance
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/:map_indexUpdates the state for single mapped task instance. *New in version 2.5.0* - PATCHUpdate the TaskInstance note.
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/:map_index/setNoteUpdate the manual user note of a mapped Task Instance. *New in version 2.5.0* - POSTList task instances (batch)
/dags/~/dagRuns/~/taskInstances/listList task instances from all DAGs and DAG runs. This endpoint is a POST to allow filtering across a large number of DAG IDs, where as a GET it would run in to…
Variable
- GETList variables
/variables?limit=100&offset=0&order_by=stringThe collection does not contain data. To get data, you must get a single entity. - POSTCreate a variable
/variables - GETGet a variable
/variables/:variable_keyGet a variable by key. - DELETEDelete a variable
/variables/:variable_key - PATCHUpdate a variable
/variables/:variable_key?update_mask=stringUpdate a variable by key.
XCom
- GETList XCom entries
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/xcomEntries?limit=100&offset=0This endpoint allows specifying ~ as the dag_id, dag_run_id, task_id to retrieve XCOM entries for for all DAGs, DAG runs and task instances. XCom values won't… - GETGet an XCom entry
/dags/:dag_id/dagRuns/:dag_run_id/taskInstances/:task_id/xcomEntries/:xcom_key?deserialize=false
Plugin
- GETGet a list of loaded plugins
/plugins?limit=100&offset=0Get a list of loaded plugins. *New in version 2.1.0*
Role
- GETList roles
/roles?limit=100&offset=0&order_by=stringGet a list of roles. *New in version 2.1.0* - POSTCreate a role
/rolesCreate a new role. *New in version 2.1.0* - GETGet a role
/roles/:role_nameGet a role. *New in version 2.1.0* - DELETEDelete a role
/roles/:role_nameDelete a role. *New in version 2.1.0* - PATCHUpdate a role
/roles/:role_name?update_mask=stringUpdate a role. *New in version 2.1.0*
Permission
- GETList permissions
/permissions?limit=100&offset=0Get a list of permissions. *New in version 2.1.0*
User
- GETList users
/users?limit=100&offset=0&order_by=stringGet a list of users. *New in version 2.1.0* - POSTCreate a user
/usersCreate a new user with unique username and email. *New in version 2.2.0* - GETGet a user
/users/:usernameGet a user with a specific username. *New in version 2.1.0* - DELETEDelete a user
/users/:usernameDelete a user with a specific username. *New in version 2.2.0* - PATCHUpdate a user
/users/:username?update_mask=stringUpdate fields for a user. *New in version 2.2.0*
DagWarning
- GETList dag warnings
/dagWarnings?dag_id=string&warning_type=string&limit=100&offset=0&order_by=string
Dataset
- GETList datasets
/datasets?limit=100&offset=0&order_by=string&uri_pattern=string - GETGet dataset events
/datasets/events?limit=100&offset=0&order_by=string&dataset_id=0&source_dag_id=string&source_task_id=string&source_run_id=string&source_map_index=0Get dataset events - GETGet a dataset
/datasets/:uriGet a dataset by uri.
API details
- Base URL
https://apache.local- Authentication
- HTTP Basic authentication (username and password)
OpenID Connect
HTTP negotiate authentication - Operations
- 73 operations on 50 pathsGET 41PATCH 13POST 12DELETE 7
- Main resources
- DAG 11TaskInstance 11DAGRun 9Connection 6Pool 5Role 5User 5Variable 5Dataset 4EventLog 2
- Response formats
application/jsontext/plainplain/text- Description format
- OpenAPI 3.0.3, API version
2.5.1 - Links
- Documentation · OpenAPI document · Support
- Listed from
- APIs.guru OpenAPI Directory (CC0-1.0)
Source updated
How to test the Airflow API (Stable) in Swastik Codes
- Open the APISelect “Fork & run in Swastik Codes” on this page. Sign in or create a free account if asked.
- Fork it into your workspaceChoose a workspace. You get your own copy of the Airflow API (Stable) collection, without the publisher's secrets.
- Add your credentialsThis API uses HTTP Basic authentication (username and password). Add yours in the collection's variables or an environment; the requests already reference it.
- Send a requestPick an endpoint, for example “Get current configuration” (GET /config), and press Send. Check the response, add tests and run the whole collection with the runner.
Frequently asked questions
What is the Airflow API (Stable)?
Overview To facilitate management, Apache Airflow supports a range of REST API endpoints across its objects. This section provides an overview of the API… It is listed under Communication and has 73 endpoints.
Who publishes the Airflow API (Stable)?
It is listed from Apache's public API description (an OpenAPI document), not published by Apache on Swastik Codes. Anyone who controls apache.org can claim the profile by adding a DNS TXT record.
What is the base URL of the Airflow API (Stable)?
Requests go to https://apache.local. Endpoint paths are added to this base URL.
How do I authenticate with the Airflow API (Stable)?
It accepts 3 methods: HTTP Basic authentication (username and password); OpenID Connect; HTTP negotiate authentication.
How many endpoints does the Airflow API (Stable) have?
73 operations on 50 paths (41 GET, 13 PATCH, 12 POST, 7 DELETE). The largest groups are DAG, TaskInstance, DAGRun, Connection.
Is forking the Airflow API (Stable) free?
Yes. Forking copies the collection into your workspace for free. It has been starred 0 times and forked 0 times.