Oracle Cloud Infrastructure

The Oracle Cloud Infrastructure integration is not supported for your selected Datadog site ().

Overview

Oracle Cloud Infrastructure (OCI) is an infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) used by enterprise-scale companies. It includes a full suite of over 30 managed services for hosting, storage, networking, databases, and more.

Use Datadog’s OCI integration to get full visibility into your OCI environment through metrics, logs, and resource data. This data enables you to power dashboards, helps with troubleshooting, and can be monitored for security and compliance posture.

Setup

Metric collection

OCI QuickStart is in Preview. Use this form to submit your request today.

Datadog’s OCI QuickStart is a fully managed, single-flow setup experience that helps you monitor your OCI infrastructure and applications in just a few clicks. OCI QuickStart creates the necessary infrastructure for forwarding metrics, logs, and resource data to Datadog, and automatically discovers new resources or OCI compartments for data collection.

Note: Only metrics are sent by default. Enable log collection and resource data collection from the Datadog OCI integration tile after completing this setup.

To set up the infrastructure for metric and log forwarding to Datadog:

The integration requires using Oracle Service Connector Hubs to forward data to Datadog. It is recommended that you request a service limit increase before completing the setup. The approximate number of Service Connector Hubs you need is:

$$\text"Service Connector Hubs" = \text"Number of compartments in tenancy" / \text"5"$$

  • Your OCI user account needs the Cloud Administrator role to complete these steps
  • You must be logged into OCI in the tenancy you want to integrate with
  • You must be logged into OCI with the Home Region selected in the top right of the screen
  • Your OCI user account needs to be in the Default Identity Domain
  • Your OCI user account must be able to create a user, user group, and dynamic group in the Default Identity Domain
  • Your OCI user account must be able to create policies in the root compartment
  • us-ashburn-1
  • ap-tokyo-1
  • sa-saopaulo-1
  • us-phoenix-1
  • eu-frankfurt-1
  • eu-stockholm-1
  • ap-singapore-1
  • us-sanjose-1
  • ca-toronto-1
  • sa-santiago-1
  • uk-london-1
  • eu-madrid-1
  • me-jeddah-1
  • us-chicago-1

Reach out through this form to request additional regions.

Datadog OCI integration tile

  1. Go to the Datadog OCI integration tile and click Add New Tenancy.
  2. Select or create a Datadog API key to use for the integration.
  3. Create a Datadog application key.
  4. Click Create OCI Stack. This takes you to an Oracle Resource Manager (ORM) stack to finish deployment.
    Note: Deploy this stack only once per tenancy.

ORM stack

  1. Accept the Oracle Terms of Use.
  2. Leave the option to use custom Terraform providers unchecked.
  3. Use the default working directory to deploy the stack in, or optionally choose a different one.
  4. Click Next, and Next again.
  5. Click Create, and wait up to 15 minutes for the deployment to complete.

To forward your OCI metrics to Datadog:

For a visual representation of this architecture, see the Architecture section.

Enter tenancy info

  • Your OCI user account needs the Cloud Administrator role to complete these steps
  • Tenancy OCID
  • Home Region

Enter the OCID and home region of the tenancy you want to monitor in the Datadog OCI integration tile.

Create OCI policy stack

  • Your OCI user account must be able to [create dynamic groups and policies][4] in the Default domain
  • You must be in the home region of the tenancy
Ensure that the home region of the tenancy is selected in the top right of the screen.

This Oracle Resource Manager (ORM) policy stack should only be deployed once per tenancy.

  1. Click the Create Policy Stack button on the Datadog OCI integration tile.
  2. Accept the Oracle Terms of Use.
  3. Leave the option to use custom Terraform providers unchecked.
  4. Use the default name and compartment for the stack, or optionally provide your own descriptive name or compartment.
  5. Click Next.
  6. Leave the tenancy field and current user field as-is.
  7. Click Next.
  8. Click Create.

Enter DatadogROAuthUser info

  • OCID of the DatadogROAuthUser
  • OCI API key and fingerprint value
  1. In the OCI console search bar, search for DatadogROAuthUser and click on the User resource that appears.
  2. Copy the user’s OCID value.
  3. Paste the value into the User OCID field in the Datadog OCI integration tile.
  4. Returning to the OCI console, generate an API key with these steps:
    a. In the bottom left corner of the screen, under Resources, click API keys.
    b. Click Add API key.
    c. Click Download private key.
    d. Click Add.
    e. A Configuration file preview popup appears, but no action is needed; close the popup.

The Add API Key page in the OCI console

  1. Copy the fingerprint value, and paste it into the Fingerprint field on the Datadog OCI integration tile.
  2. Copy the private key value with these steps: a. Open the downloaded private key .pem file in a text editor, or use a terminal command such as cat to display the file’s contents. b. Copy the entire contents, including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----.
  3. Paste the private key value into the Private Key field on the Datadog OCI integration tile.

Create OCI metric forwarding stack

  • Your OCI user account must be able to create resources in the compartment
  • [Datadog API Key][6] value
  • Username and auth token for a user with the REPOSITORY_READ and REPOSITORY_UPDATE permissions to pull and push images to a Docker repo
    • See [Getting an Auth Token][7] to find out how to create an auth token
    • See [Policies to Control Repository Access][8] for more information on the required policies

Note: To verify the Docker registry login is correct, see [Logging in to Oracle Cloud Infrastructure Registry][9].

The metric forwarding stack must be deployed for each combination of tenancy and region to be monitored. For the simplest setup, Datadog recommends creating all the necessary OCI resources with the Oracle Resource Manager (ORM) stack provided below. Alternatively, you can use your existing OCI networking infrastructure.

All resources created by Datadog’s ORM stack are deployed to the compartment specified, and for the region currently selected in the top right of the screen.

  1. Click the Create Metric Stack button on the Datadog OCI integration tile.
  2. Accept the Oracle Terms of Use.
  3. Leave the Custom providers option unchecked.
  4. Name the stack and select the compartment to deploy it to.
  5. Click Next.
  6. In the Datadog API Key field, enter your Datadog API key value.
  7. In the Network options section, leave Create VCN checked.

If using an existing Virtual Cloud Network (VCN), the subnet’s OCID must be provided to the stack. Make sure that the VCN:

  • Is allowed to make HTTP egress calls through NAT gateway
  • Is able to pull images from OCI container registry using service gateway
  • Has the route table rules to allow NAT gateway and service gateway
  • Has the security rules to send HTTP requests
  1. In the Network options section, uncheck the Create VCN option and enter your VCN information:
    a. In the vcnCompartment field, select your compartment.
    b. In the existingVcn section, select your existing VCN.
    c. In the Function Subnet OCID section, enter the OCID of the subnet to be used.
  1. In the Metrics settings section, optionally remove any metric namespaces from collection.
  2. In the Metrics compartments section, enter a comma-separated list of compartment OCIDs to monitor. Any metric namespace filters selected in the previous step are applied to each compartment.
  3. In the Function settings section, select GENERIC_ARM. Select GENERIC_X86 if deploying in a Japan region.
  4. Click Next.
  5. Click Create.
  6. Return to the Datadog OCI integration tile and click Create Configuration.

Notes:

  • By default, only the root compartment is selected, and all of the metric namespaces from Step 8 which are present in the compartment are enabled (up to 50 namespaces are supported per connector hub). If you choose to monitor additional compartments, the namespaces added to them are an intersection of namespaces selected and the namespaces present in the compartment.
  • You should manage who has access to the Terraform state files of the resource manager stacks. See the Terraform State Files section of the Securing Resource Manager page for more information.

Complete the setup in Datadog

Return to the Datadog OCI integration tile and click Ready!.

Validation

View oci.* metrics in the OCI integration overview dashboard or Metrics Explorer page in Datadog.

OCI function metrics (oci.faas namespace) and container instance metrics (oci_computecontainerinstance namespace) are in Preview.

Configuration

The configuration tab of an OCI tenancy in Datadog

After completing the setup, a configuration tab for the tenancy becomes available on the left side of the Datadog OCI integration tile. Apply tenancy-wide data collection configurations as outlined in the sections below.

Add regions

On the General tab, select the regions for data collection from the Regions checkbox list. Region selections apply to the entire tenancy, for both metrics and logs.

Note: If you used the QuickStart setup method, and afterward subscribed to a new OCI region, reapply the initial setup stack in ORM. The new region then becomes available in the Datadog OCI tile.

Metric and log collection

Use the Metric collection and Log collection tabs to configure which metrics and logs are sent to Datadog:

  • Enable or disable collection of metrics or logs for the entire tenancy
  • Include or exclude specific compartments based on key:value format compartment tags. For example:
    • datadog:monitored,env:prod* includes compartments if either of these tags is present
    • !env:staging,!testing excludes compartments only if both tags are present
    • datadog:monitored,!region:us-phoenix-1 includes compartments that both have the tag datadog:monitored and do not have the tag region:us-phoenix-1
  • Enable or disable collection for specific OCI services

Notes:

  • After modifying tags in OCI, it may take up to 15 minutes for the changes to appear in Datadog
  • In OCI, tags are not inherited by child compartments; each compartment must be tagged individually

Log collection

Use one of the methods below to send your OCI logs to Datadog:

  1. Follow the steps in the setup section to create the necessary infrastructure for forwarding both metrics and logs to Datadog.
  2. Click the Enable Log Collection toggle on the Log Collection tab of the Datadog OCI integration tile.
  1. Configure an OCI log.
  2. Create an OCI function.
  3. Setup an OCI Service Connector.

The instructions below use the OCI portal to set up the integration.

OCI logging

  1. In the OCI portal, navigate to Logging -> Log Groups.
  2. Select your compartment and click Create Log Group. A side panel opens.
  3. Enter data_log_group for the name, and optionally provide a description and tags.
  4. Click Create to set up your new Log Group.
  5. Under Resources, click Logs.
  6. Click to Create custom log or Enable service log as desired.
  7. Click Enable Log, to create your new OCI Log.

For more information on OCI Logs, see Enabling Logging for a Resource.

OCI function

  1. In the OCI portal, navigate to Functions.
  2. Select an existing application or click Create Application.
  3. Create a new OCI function within your application. See the Oracle Overview of Functions for details.
  4. It is recommended to create a boilerplate Python function first and replace the automatically-generated files with Datadog’s source code:
    • Replace func.py with code from the Datadog OCI repo
    • Replace func.yaml with code from the Datadog OCI repo. DATADOG_TOKEN and DATADOG_HOST must be replaced with your Datadog API key and region logs intake link
    • Replace requirements.txt with code from the Datadog OCI repo

OCI service connector hub

  1. In the OCI portal, navigate to Logging -> Service Connectors.
  2. Click Create Service Connector to be directed to the Create Service Connector page.
  3. Select the Source as Logging and Target as Functions.
  4. Under Configure Source Connection select a Compartment name, Log Group, and Log. (The Log Group and Log created in the first step)
  5. If you also want to send Audit Logs, click +Another Log and select the same Compartment while replacing “_Audit” as your Log Group.
  6. Under Configure target select a Compartment, Function application, and Function. (The Function Application and Function created in the previous step)
  7. If you are prompted to create a policy, click Create from the prompt.
  8. Click Create at the bottom to finish creating your Service Connector.

For more information on OCI Object Storage, see Oracle’s Service Connector blog post.

  1. Configure an OCI log.
  2. Create an OCI object store and enable read/write access for OCI logs.
  3. Create an OCI function.
  4. Set up an OCI event.

The instructions below use the OCI portal to set up the integration.

OCI logging

  1. In the OCI portal, navigate to Solutions and Platform -> Logging -> Logs.
  2. Click Create Custom Log to be directed to the Create Custom Log page.
  3. Give your new OCI log a name.
  4. Select a Compartment and Log Group. These selections remain consistent across the entire installation.
  5. Click Create Custom Log to be directed to the Create Agent Config page.
  6. Click Create new configuration.
  7. Give your new configuration a name. Your compartment is preselected for you.
  8. Set the group type to Dynamic Group and group to one of your existing groups.
  9. Set the input type to Log Path, enter your preferred input name and use “/” for file paths.
  10. Click Create Custom Log, then your OCI log is created and available on the logs page.

For more information on OCI Logs, see Enabling Logging for a Resource.

OCI object storage

  1. In the OCI portal, navigate to Core Infrastructure -> Object Storage -> Object Storage.
  2. Click Create Bucket to be directed to the Create Bucket form.
  3. Select Standard for your storage tier and check Emit Object Events.
  4. Complete the rest of the form based on your preference.
  5. Click Create Bucket, then your bucket is created and available in the bucket list.
  6. Select your new bucket from the active bucket list and click Logs under resources.
  7. Toggle read to enabled, which directs you to an Enable Log side menu.
  8. Select a Compartment and Log Group (use the same selections as your OCI log).
  9. Enter a name for the Log Name and select your preferred log retention.

For more information on OCI Object Storage, see Putting Data into Object Storage.

OCI function

  1. In the OCI portal, navigate to Solutions and Platform -> Developer Services -> Functions.
  2. Select an existing application or click Create Application.
  3. Create a new OCI function within your application. See the Oracle Overview of Functions for more details.
  4. It is recommended to create a boilerplate Python function first and replace the automatically-generated files with Datadog’s source code:
    • Replace func.py with code from the Datadog OCI repo
    • Replace func.yaml with code from the Datadog OCI repo. DATADOG_TOKEN and DATADOG_HOST must be replaced with your Datadog API key and region logs intake link
    • Replace requirements.txt with code from the Datadog OCI repo

OCI event

  1. In the OCI portal, navigate to Solutions and Platform -> Application Integration -> Event Service.
  2. Click Create Rule to be directed to the Create Rule page.
  3. Give your event rule a name and description.
  4. Set your condition as Event Type, service name as Object Storage, and event type as Object - Create.
  5. Set your action type as Functions.
  6. Ensure that your function compartment is the same selection you made for OCI Log, OCI Bucket, and OCI Function.
  7. Select your function application and function (according to the previous installation step).
  8. Click Create Rule, then your rule is created and available in the rules list.

For more information on OCI Object Storage, see Getting Started with Events.

Resource Collection

On the Resource Collection tab of the Datadog OCI integration tile, click the Enable Resource Collection toggle. Resources are visible in the Datadog Resource Catalog.

Architecture

Metric and log forwarding resources

A diagram of the OCI metric and log forwarding resources mentioned for this setup option and displaying the flow of data

For each region monitored, this setup option creates the following infrastructure within that region to forward metrics and logs to Datadog:

  • Function Application (dd-function-app)
  • Two functions:
    • Metrics Forwarder (dd-metrics-forwarder)
    • Logs Forwarder (dd-logs-forwarder)
  • VCN (dd-vcn) with secure networking infrastructure:
    • Private subnet (dd-vcn-private-subnet)
    • NAT gateway (dd-vcn-natgateway) for external access to the internet
    • Service gateway (dd-vcn-servicegateway) for internal access to OCI services
  • Key Management Service (KMS) vault (datadog-vault) to store the Datadog API key
  • Dedicated Datadog compartment (Datadog)

All resources are tagged with ownedby = "datadog".

IAM resources

A diagram of the OCI IAM resources mentioned for this setup option and displaying the flow of data

This setup option creates the following IAM resources to enable data forwarding to Datadog:

  • Service user (dd-svc)
  • Group (dd-svc-admin) that the service user belongs to
  • RSA key pair for API authentication
  • OCI API key for the service user
  • Dynamic Group (dd-dynamic-group-connectorhubs) that includes all service connectors in the Datadog compartment
  • Dynamic Group (dd-dynamic-group-function) that includes all functions in the Datadog compartment
  • Policy (dd-svc-policy) to give the service user read access to the tenancy resources, as well as access to manage OCI Service Connector Hubs and OCI Functions in the compartment created and managed by Datadog
- Allow dd-svc-admin to read all-resources in tenancy
- Allow dd-svc-admin to manage serviceconnectors in Datadog compartment
- Allow dd-svc-admin to manage functions-family in Datadog compartment with specific permissions:
     * FN_FUNCTION_UPDATE
     * FN_FUNCTION_LIST
     * FN_APP_LIST
- Endorse dd-svc-admin to read objects in tenancy usage-report
  • Policy dd-dynamic-group-policy to enable the service connectors to read data (logs and metrics) and interact with functions. This policy also allows the functions to read secrets in the Datadog compartment (the Datadog API and application keys stored in the KMS vault)
   - Allow dd-dynamic-group-connectorhubs to read log-content in tenancy
   - Allow dd-dynamic-group-connectorhubs to read metrics in tenancy
   - Allow dd-dynamic-group-connectorhubs to use fn-function in Datadog compartment
   - Allow dd-dynamic-group-connectorhubs to use fn-invocation in Datadog compartment
   - Allow dd-dynamic-group-functions to read secret-bundles in Datadog compartment

Metric forwarding resources

A diagram of the OCI resources mentioned for this setup option and displaying the flow of data

This setup option creates an OCI connector hub, function app, and secure networking infrastructure to forward OCI metrics to Datadog. The ORM stack for these resources creates a function container repository for the region in the tenancy, and the Docker image is pushed to it to be used by the function.

IAM resources

A diagram of the OCI resources and workflow used for integration authentication

This setup option creates:

  • Dynamic group with resource.type = 'serviceconnectors', to enable access to the connector hub
  • User named DatadogROAuthUser, which Datadog uses to read tenancy resources
  • Group to which the created user is added for policy access
  • User named DatadogAuthWriteUser, which is used to push Docker images for the function
  • Write access group that the DatadogAuthWriteUser is added to, for pushing images through policy access
  • Policy in the root compartment to allow connector hubs to read metrics and invoke functions. This policy also gives the created user group read access to both the tenancy resources and write access group, to push images
Allow dynamic-group Default/<GROUP_NAME> to read metrics in tenancy
Allow dynamic-group Default/<GROUP_NAME> to use fn-function in tenancy
Allow dynamic-group Default/<GROUP_NAME> to use fn-invocation in tenancy
Allow group Default/<USER_GROUP_NAME> to read all-resources in tenancy
Allow group Default/<WRITE_USER_GROUP_NAME> to manage repos in tenancy where ANY {request.permission = 'REPOSITORY_READ', request.permission = 'REPOSITORY_UPDATE', request.permission = 'REPOSITORY_CREATE'}

Data Collected

Metrics

See metadata.csv for a list of metrics provided by this integration.

Service Checks

The OCI integration does not include any service checks.

Events

The OCI integration does not include any events.

Troubleshooting

Need help? Contact Datadog support.

Further Reading

Additional helpful documentation, links, and articles:

PREVIEWING: esther/docs-11020-sheets-update