- 필수 기능
- 시작하기
- Glossary
- 표준 속성
- Guides
- Agent
- 통합
- 개방형텔레메트리
- 개발자
- API
- Datadog Mobile App
- CoScreen
- Cloudcraft
- 앱 내
- 서비스 관리
- 인프라스트럭처
- 애플리케이션 성능
- APM
- Continuous Profiler
- 스팬 시각화
- 데이터 스트림 모니터링
- 데이터 작업 모니터링
- 디지털 경험
- 소프트웨어 제공
- 보안
- AI Observability
- 로그 관리
- 관리
Supported languages:
Language | Version |
---|---|
Ruby | >= 2.7 |
JRuby | >= 9.4 |
Supported test frameworks:
Test Framework | Version |
---|---|
RSpec | >= 3.0.0 |
Minitest | >= 5.0.0 |
Cucumber | >= 3.0 |
Supported test runners:
Test runner | Version |
---|---|
Knapsack Pro | >= 7.2.0 |
ci-queue | >= 0.53.0 |
To report test results to Datadog, you need to configure the datadog-ci
gem:
If you are using a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode. For this, set the following environment variables:
DD_CIVISIBILITY_AGENTLESS_ENABLED=true
(Required)false
DD_API_KEY
(Required)(empty)
Additionally, configure the Datadog site to which you want to send data.
DD_SITE
(Required)datadoghq.com
If you are running tests on an on-premises CI provider, such as Jenkins or self-managed GitLab CI, install the Datadog Agent on each worker node by following the Agent installation instructions. This is the recommended option as it allows you to automatically link test results to logs and underlying host metrics.
If you are using a Kubernetes executor, Datadog recommends using the Datadog Operator. The operator includes Datadog Admission Controller which can automatically inject the tracer library into the build pods. Note: If you use the Datadog Operator, there is no need to download and inject the tracer library since the Admission Controller can do this for you, so you can skip the corresponding step below. However, you still need to make sure that your pods set the environment variables or command-line parameters necessary to enable Test Visibility.
If you are not using Kubernetes or can’t use the Datadog Admission Controller and the CI provider is using a container-based executor, set the DD_TRACE_AGENT_URL
environment variable (which defaults to http://localhost:8126
) in the build container running the tracer to an endpoint that is accessible from within that container. Note: Using localhost
inside the build references the container itself and not the underlying worker node or any container where the Agent might be running in.
DD_TRACE_AGENT_URL
includes the protocol and port (for example, http://localhost:8126
) and takes precedence over DD_AGENT_HOST
and DD_TRACE_AGENT_PORT
, and is the recommended configuration parameter to configure the Datadog Agent’s URL for CI Visibility.
If you still have issues connecting to the Datadog Agent, use the Agentless Mode. Note: When using this method, tests are not correlated with logs and infrastructure metrics.
To install the Ruby test visibility library:
datadog-ci
gem to your Gemfile
:Gemfile
source "<https://rubygems.org>"
gem "datadog-ci", "~> 1.0", group: :test
bundle install
The RSpec integration traces all executions of example groups and examples when using the rspec
test framework.
To activate your integration, add this to the spec_helper.rb
file:
require "rspec"
require "datadog/ci"
# Only activates test instrumentation on CI
if ENV["DD_ENV"] == "ci"
Datadog.configure do |c|
# enables test visibility
c.ci.enabled = true
# The name of the service or library under test
c.service = "my-ruby-app"
# Enables the RSpec instrumentation
c.ci.instrument :rspec
end
end
Run your tests as you normally do, specifying the environment where tests are being run in the DD_ENV
environment variable.
You could use the following environments:
local
when running tests on a developer workstationci
when running them on a CI providerFor example:
DD_ENV=ci bundle exec rake spec
The Minitest integration traces all executions of tests when using the minitest
framework.
To activate your integration, add this to the test_helper.rb
file:
require "minitest"
require "datadog/ci"
# Only activates test instrumentation on CI
if ENV["DD_ENV"] == "ci"
Datadog.configure do |c|
# enables test visibility
c.ci.enabled = true
# The name of the service or library under test
c.service = "my-ruby-app"
c.ci.instrument :minitest
end
end
Run your tests as you normally do, specifying the environment where tests are being run in the DD_ENV
environment variable.
You could use the following environments:
local
when running tests on a developer workstationci
when running them on a CI providerFor example:
DD_ENV=ci bundle exec rake test
Example configuration with minitest/autorun
:
require "datadog/ci"
require "minitest/autorun"
if ENV["DD_ENV"] == "ci"
Datadog.configure do |c|
c.ci.enabled = true
c.service = "my-ruby-app"
c.ci.instrument :minitest
end
end
The Cucumber integration traces executions of scenarios and steps when using the cucumber
framework.
To activate your integration, add the following code to your application:
require "cucumber"
require "datadog/ci"
# Only activates test instrumentation on CI
if ENV["DD_ENV"] == "ci"
Datadog.configure do |c|
# enables test visibility
c.ci.enabled = true
# The name of the service or library under test
c.service = "my-ruby-app"
# Enables the Cucumber instrumentation
c.ci.instrument :cucumber
end
end
Run your tests as you normally do, specifying the environment where tests are being run in the DD_ENV
environment variable.
You could use the following environments:
local
when running tests on a developer workstationci
when running them on a CI providerFor example:
DD_ENV=ci bundle exec rake cucumber
You can add custom tags to your tests by using the current active test:
require "datadog/ci"
# inside your test
Datadog::CI.active_test&.set_tag("test_owner", "my_team")
# test continues normally
# ...
To create filters or group by
fields for these tags, you must first create facets. For more information about adding tags, see the Adding Tags section of the Ruby custom instrumentation documentation.
Like tags, you can add custom measures to your tests by using the current active test:
require "datadog/ci"
# inside your test
Datadog::CI.active_test&.set_metric("memory_allocations", 16)
# test continues normally
# ...
For more information on custom measures, see the Add Custom Measures Guide.
The following is a list of the most important configuration settings that can be used with the test visibility library, either in code by using a Datadog.configure
block, or using environment variables:
service
DD_SERVICE
$PROGRAM_NAME
my-ruby-app
env
DD_ENV
none
local
, ci
For more information about service
and env
reserved tags, see Unified Service Tagging.
The following environment variable can be used to configure the location of the Datadog Agent:
DD_TRACE_AGENT_URL
http://hostname:port
.http://localhost:8126
All other Datadog Tracer configuration options can also be used.
It can be useful to have rich tracing information about your tests that includes time spent performing database operations or other external calls, as seen in the following flame graph:
To achieve this, configure additional instrumentation in your configure
block:
if ENV["DD_ENV"] == "ci"
Datadog.configure do |c|
# ... ci configs and instrumentation here ...
c.tracing.instrument :redis
c.tracing.instrument :pg
# ... any other instrumentations supported by datadog gem ...
end
end
Alternatively, you can enable automatic instrumentation in test_helper/spec_helper
:
require "datadog/auto_instrument" if ENV["DD_ENV"] == "ci"
Note: In CI mode, these traces are submitted to CI Visibility, and they do not show up in Datadog APM.
For the full list of available instrumentation methods, see the tracing documentation
Webmock and VCR are popular Ruby libraries that stub HTTP requests when running tests. By default, they fail when used with datadog-ci because traces are being sent to Datadog with HTTP calls.
To allow HTTP connections for Datadog backend, you need to configure Webmock and VCR accordingly.
# Webmock
# when using Agentless mode:
WebMock.disable_net_connect!(:allow => /datadoghq/)
# when using Agent running locally:
WebMock.disable_net_connect!(:allow_localhost => true)
# or for more granular setting set your Agent URL, for example:
WebMock.disable_net_connect!(:allow => "localhost:8126")
# VCR
VCR.configure do |config|
# ... your usual configuration here ...
# when using Agent
config.ignore_hosts "127.0.0.1", "localhost"
# when using Agentless mode
config.ignore_request do |request|
# ignore all requests to datadoghq hosts
request.uri =~ /datadoghq/
end
end
Datadog uses Git information for visualizing your test results and grouping them by repository, branch, and commit. Git metadata is automatically collected by the test instrumentation from CI provider environment variables and the local .git
folder in the project path, if available.
If you are running tests in non-supported CI providers or with no .git
folder, you can set the Git information manually using environment variables. These environment variables take precedence over any auto-detected information. Set the following environment variables to provide Git information:
DD_GIT_REPOSITORY_URL
git@github.com:MyCompany/MyApp.git
, https://github.com/MyCompany/MyApp.git
DD_GIT_BRANCH
develop
DD_GIT_TAG
1.0.1
DD_GIT_COMMIT_SHA
a18ebf361cc831f5535e58ec4fae04ffd98d8152
DD_GIT_COMMIT_MESSAGE
Set release number
DD_GIT_COMMIT_AUTHOR_NAME
John Smith
DD_GIT_COMMIT_AUTHOR_EMAIL
john@example.com
DD_GIT_COMMIT_AUTHOR_DATE
2021-03-12T16:00:28Z
DD_GIT_COMMIT_COMMITTER_NAME
Jane Smith
DD_GIT_COMMIT_COMMITTER_EMAIL
jane@example.com
DD_GIT_COMMIT_COMMITTER_DATE
2021-03-12T16:00:28Z
If you use RSpec, Minitest, or Cucumber, do not use the manual testing API, as CI Visibility automatically instruments them and sends the test results to Datadog. The manual testing API is incompatible with already supported testing frameworks.
Use the manual testing API only if you use an unsupported testing framework or have a different testing mechanism. Full public API documentation is available on YARD site.
The API is based around four concepts: test session, test module, test suite, and test.
A test session represents a test command run.
To start a test session, call Datadog::CI.start_test_session
and pass the Datadog service and tags (such as the test framework
you are using).
When all your tests have finished, call Datadog::CI::TestSession#finish
, which closes the session and sends the session
trace to the backend.
A test module represents a smaller unit of work within a session. For supported test frameworks, test module is always same as test session. For your use case, this could be a package in your componentized application.
To start a test module, call Datadog::CI.start_test_module
and pass the name of the module.
When the module run has finished, call Datadog::CI::TestModule#finish
.
A test suite comprises a set of tests that test similar functionality. A single suite usually corresponds to a single file where tests are defined.
Create test suites by calling Datadog::CI#start_test_suite
and passing the name of the test suite.
Call Datadog::CI::TestSuite#finish
when all the related tests in the suite have finished their execution.
A test represents a single test case that is executed as part of a test suite. Usually it corresponds to a method that contains testing logic.
Create tests in a suite by calling Datadog::CI#start_test
or Datadog::CI.trace_test
and passing the name of the test and name of the test suite. Test suite name must be the same as name of the test suite started in previous step.
Call Datadog::CI::Test#finish
when a test has finished execution.
The following code represents example usage of the API:
require "datadog/ci"
Datadog.configure do |c|
c.service = "my-test-service"
c.ci.enabled = true
end
def run_test_suite(tests, test_suite_name)
test_suite = Datadog::CI.start_test_suite(test_suite_name)
run_tests(tests, test_suite_name)
test_suite.passed!
test_suite.finish
end
def run_tests(tests, test_suite_name)
tests.each do |test_name|
Datadog::CI.trace_test(test_name, test_suite_name) do |test|
test.passed!
end
end
end
Datadog::CI.start_test_session(
tags: {
Datadog::CI::Ext::Test::TAG_FRAMEWORK => "my-framework",
Datadog::CI::Ext::Test::TAG_FRAMEWORK_VERSION => "0.0.1",
}
)
Datadog::CI.start_test_module("my-test-module")
run_test_suite(["test1", "test2", "test3"], "test-suite-name")
Datadog::CI.active_test_module&.passed!
Datadog::CI.active_test_module&.finish
Datadog::CI.active_test_session&.passed!
Datadog::CI.active_test_session&.finish