Datadog (dagster-datadog)¶

This library provides an integration with Datadog, to support publishing metrics to Datadog from within Dagster ops.

We use the Python datadogpy library. To use it, you’ll first need to create a DataDog account and get both API and Application keys.

The integration uses DogStatsD, so you’ll need to ensure the datadog agent is running on the host you’re sending metrics from.

class dagster_datadog.DatadogResource(*, api_key, app_key)[source]¶

This resource is a thin wrapper over the dogstatsd library.

As such, we directly mirror the public API methods of DogStatsd here; you can refer to the DataDog documentation for how to use this resource.

Examples

@op
def datadog_op(datadog_client: ResourceParam[DatadogClient]):
    datadog_client.event('Man down!', 'This server needs assistance.')
    datadog_client.gauge('users.online', 1001, tags=["protocol:http"])
    datadog_client.increment('page.views')
    datadog_client.decrement('page.views')
    datadog_client.histogram('album.photo.count', 26, tags=["gender:female"])
    datadog_client.distribution('album.photo.count', 26, tags=["color:blue"])
    datadog_client.set('visitors.uniques', 999, tags=["browser:ie"])
    datadog_client.service_check('svc.check_name', datadog_client.WARNING)
    datadog_client.timing("query.response.time", 1234)

    # Use timed decorator
    @datadog_client.timed('run_fn')
    def run_fn():
        pass

    run_fn()

@job
def job_for_datadog_op() -> None:
    datadog_op()

job_for_datadog_op.execute_in_process(
    resources={"datadog_client": DatadogResource(api_key="FOO", app_key="BAR")}
)

dagster_datadog.datadog_resource ResourceDefinition[source]¶

Config Schema:

api_key (dagster.StringSource):

Datadog API key. See https://docs.datadoghq.com/account_management/api-app-keys/

app_key (dagster.StringSource):

Datadog application key. See https://docs.datadoghq.com/account_management/api-app-keys/.

This legacy resource is a thin wrapper over the dogstatsd library.

Prefer using DatadogResource.

As such, we directly mirror the public API methods of DogStatsd here; you can refer to the DataDog documentation for how to use this resource.

Examples

@op(required_resource_keys={'datadog'})
def datadog_op(context):
    dd = context.resources.datadog

    dd.event('Man down!', 'This server needs assistance.')
    dd.gauge('users.online', 1001, tags=["protocol:http"])
    dd.increment('page.views')
    dd.decrement('page.views')
    dd.histogram('album.photo.count', 26, tags=["gender:female"])
    dd.distribution('album.photo.count', 26, tags=["color:blue"])
    dd.set('visitors.uniques', 999, tags=["browser:ie"])
    dd.service_check('svc.check_name', dd.WARNING)
    dd.timing("query.response.time", 1234)

    # Use timed decorator
    @dd.timed('run_fn')
    def run_fn():
        pass

    run_fn()

@job(resource_defs={'datadog': datadog_resource})
def dd_job():
    datadog_op()

result = dd_job.execute_in_process(
    run_config={'resources': {'datadog': {'config': {'api_key': 'YOUR_KEY', 'app_key': 'YOUR_KEY'}}}}
)