Oracle Cloud Infrastructure
Overview
Oracle Cloud Infrastructure (OCI) is an infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) used by enterprise-scale companies. With a full suite of managed services for hosting, storage, networking, databases, and more.
Use Datadog’s OCI integration to forward your logs and metrics to Datadog, where they can power dashboards, help with troubleshooting, and be monitored for security and compliance posture.
Setup
Metric collection
To forward your OCI metrics to Datadog:
For a visual representation of this architecture, see the Architecture section.
Enter tenancy info
Enter the OCID and home region of the tenancy you want to monitor in the Datadog OCI integration tile.
Create OCI policy stack
Ensure that the home region of the tenancy is selected in the top right of the screen.
This policy stack should only be deployed once per tenancy.
- Click the Create Policy Stack button on the Datadog OCI integration tile.
- Accept the Oracle Terms of Use.
- Leave the option to use custom Terraform providers unchecked.
- Use the default name and compartment for the stack, or optionally provide your own descriptive name or compartment.
- Click Next.
- Leave the tenancy field and current user field as-is.
- Click Next.
- Click Create.
Enter DatadogROAuthUser info
- In the OCI console search bar, search for
DatadogROAuthUser and click on the User resource that appears. - Copy the user’s OCID value.
- Paste the value into the User OCID field in the Datadog OCI integration tile.
- Returning to the OCI console, generate an API key with these steps:
a. In the bottom left corner of the screen, under Resources, click API keys.
b. Click Add API key.
c. Click Download private key.
d. Click Add.
e. A Configuration file preview popup appears, but no action is needed; close the popup.
- Copy the fingerprint value, and paste it into the Fingerprint field on the Datadog OCI integration tile.
- Copy the private key value with these steps:
a. Open the downloaded private key
.pem file in a text editor, or use a terminal command such as cat to display the file’s contents.
b. Copy the entire contents, including -----BEGIN PRIVATE KEY----- and -----END PRIVATE KEY-----. - Paste the private key value into the Private Key field on the Datadog OCI integration tile.
Create OCI metric forwarding stack
- Your user account must be able to create resources in the compartment.
- Datadog API Key value
- Username and auth token for a user with the
REPOSITORY_READ and REPOSITORY_UPDATE permissions to pull and push images to a Docker repo.
Note: To verify the Docker registry login is correct, see Logging in to Oracle Cloud Infrastructure Registry.
The metric forwarding stack must be deployed for each combination of tenancy and region to be monitored. For the simplest setup, Datadog recommends creating all the necessary OCI resources with the ORM stack provided below. Alternatively, you can use your existing OCI networking infrastructure.
All resources created by Datadog’s ORM stack are deployed to the compartment specified, and for the region currently selected in the top right of the screen.
- Click the Create Metric Stack button on the Datadog OCI integration tile.
- Accept the Oracle Terms of Use.
- Leave the Custom providers option unchecked.
- Name the stack and select the compartment to deploy it to.
- Click Next.
- In the Datadog API Key field, enter your Datadog API key value.
- In the Network options section, leave
Create VCN checked.
If using an existing VCN, the subnet’s OCID must be provided to the stack. Make sure that the VCN:
- Is allowed to make HTTP egress calls through NAT gateway.
- Is able to pull images from OCI container registry using service gateway.
- Has the route table rules to allow NAT gateway and service gateway.
- Has the security rules to send HTTP requests.
- In the Network options section, uncheck the
Create VCN option and enter your VCN information:
a. In the vcnCompartment field, select your compartment.
b. In the existingVcn section, select your existing VCN.
c. In the Function Subnet OCID section, enter the OCID of the subnet to be used.
- In the Metrics settings section, optionally remove any metric namespaces from collection.
- In the Metrics compartments section, enter a comma-separated list of compartment OCIDs to monitor. Any metric namespace filters selected in the previous step are applied to each compartment.
- In the Function settings section, select GENERIC_ARM. Select GENERIC_X86 if deploying in a Japan region.
- Click Next.
- Click Create.
- Return to the Datadog OCI integration tile and click Create Configuration.
Notes:
- By default, only the root compartment is selected, and all of the metric namespaces supported by the Datadog OCI integration are enabled (up to 50 namespaces are supported per connector hub). If you choose to monitor additional compartments, any metric namespace exclusion filters are applied to each compartment.
- You should manage who has access to the Terraform state files of the resource manager stacks. See the Terraform State Files section of the Securing Resource Manager page for more information.
Validation
View oci.* metrics in the OCI integration overview dashboard or Metrics Explorer page in Datadog.
OCI function metrics (oci.faas namespace) and container instance metrics (oci_computecontainerinstance namespace) are in Preview.
Configuration
Add regions
To monitor an additional region in a tenancy, navigate to that tenancy in the OCI Integration tile.
- In the Configure an Additional Region section, click Create Metric Stack.
- Switch to the region you wish to monitor in the top right of the screen.
- Complete the steps in Create OCI metric forwarding stack for the new region.
Add compartments or metric namespaces
To add compartments or edit the list of metric namespaces enabled, click Edit on the newly created Connector Hub.
- Click + Another compartment to add compartments.
- In the Configure source section, add or remove namespaces from the Namespaces dropdown.
Architecture
Metric forwarding resources
This integration creates an OCI connector hub, function app, and secure networking infrastructure to forward OCI metrics to Datadog. The ORM stack for these resources creates a function container repository for the region in the tenancy, and the Docker image is pushed to it to be used by the function.
IAM resources
This integration creates:
- A dynamic group with
resource.type = 'serviceconnectors', to enable access to the connector hub. - A user called DatadogROAuthUser, which Datadog uses to read tenancy resources.
- A group to which the created user is added for policy access.
- A user called DatadogAuthWriteUser, which is used to push Docker images for the function.
- A write access group that the
DatadogAuthWriteUser is added to, for pushing images through policy access. - A policy in the root compartment to allow connector hubs to read metrics and invoke functions. This policy also gives the created user group read access to both the tenancy resources and write access group, to push images. The following statements are added to the policy:
Allow dynamic-group Default/<GROUP_NAME> to read metrics in tenancy
Allow dynamic-group Default/<GROUP_NAME> to use fn-function in tenancy
Allow dynamic-group Default/<GROUP_NAME> to use fn-invocation in tenancy
Allow group Default/<USER_GROUP_NAME> to read all-resources in tenancy
Allow group Default/<WRITE_USER_GROUP_NAME> to manage repos in tenancy where ANY {request.permission = 'REPOSITORY_READ', request.permission = 'REPOSITORY_UPDATE', request.permission = 'REPOSITORY_CREATE'}
Metric namespaces
Log collection
Send logs from your Oracle Cloud Infrastructure to Datadog by following either process:
- Configure an OCI log.
- Create an OCI function.
- Setup an OCI Service Connector.
The instructions below use the OCI portal to set up the integration.
OCI logging
- In the OCI portal, navigate to Logging -> Log Groups.
- Select your compartment and click Create Log Group. A side panel opens.
- Enter
data_log_group for the name, and optionally provide a description and tags. - Click Create to set up your new Log Group.
- Under Resources, click Logs.
- Click to Create custom log or Enable service log as desired.
- Click Enable Log, to create your new OCI Log.
For more information on OCI Logs, see Enabling Logging for a Resource.
OCI function
- In the OCI portal, navigate to Functions.
- Select an existing application or click Create Application.
- Create a new OCI function within your application. See the Oracle Overview of Functions for details.
- It is recommended to create a boilerplate Python function first and replace the auto generated files with Datadog’s source code:
- Replace
func.py with code from the Datadog OCI repo. - Replace
func.yaml with code from the Datadog OCI repo. DATADOG_TOKEN and DATADOG_HOST must be replaced with your Datadog API key and region logs intake link. - Replace
requirements.txt with code from the Datadog OCI repo.
OCI service connector hub
- In the OCI portal, navigate to Logging -> Service Connectors.
- Click Create Service Connector to be directed to the Create Service Connector page.
- Select the Source as Logging and Target as Functions.
- Under Configure Source Connection select a Compartment name, Log Group, and Log. (The Log Group and Log created in the first step)
- If you also want to send Audit Logs, click +Another Log and select the same Compartment while replacing “_Audit” as your Log Group.
- Under Configure target select a Compartment, Function application, and Function. (The Function Application and Function created in the previous step)
- If you are prompted to create a policy, click Create from the prompt.
- Click Create at the bottom to finish creating your Service Connector.
For more information on OCI Object Storage, see Oracle’s Service Connector blog post.
- Configure an OCI log.
- Create an OCI object store and enable read/write access for OCI logs.
- Create an OCI function.
- Set up an OCI event.
The instructions below use the OCI portal to set up the integration.
OCI logging
- In the OCI portal, navigate to Solutions and Platform -> Logging -> Logs.
- Click Create Custom Log to be directed to the Create Custom Log page.
- Give your new OCI log a name.
- Select a Compartment and Log Group. These selections remain consistent across the entire installation.
- Click Create Custom Log to be directed to the Create Agent Config page.
- Click Create new configuration.
- Give your new configuration a name. Your compartment is preselected for you.
- Set the group type to Dynamic Group and group to one of your existing groups.
- Set the input type to Log Path, enter your preferred input name and use “/” for file paths.
- Click Create Custom Log, then your OCI log is created and available on the logs page.
For more information on OCI Logs, see Enabling Logging for a Resource.
OCI object storage
- In the OCI portal, navigate to Core Infrastructure -> Object Storage -> Object Storage.
- Click Create Bucket to be directed to the Create Bucket form.
- Select Standard for your storage tier and check Emit Object Events.
- Complete the rest of the form based on your preference.
- Click Create Bucket, then your bucket is created and available in the bucket list.
- Select your new bucket from the active bucket list and click Logs under resources.
- Toggle read to enabled which directs you to an Enable Log side menu.
- Select a Compartment and Log Group (use the same selections as your OCI log).
- Enter a name for the Log Name and select your preferred log retention.
For more information on OCI Object Storage, see Putting Data into Object Storage.
OCI function
- In the OCI portal, navigate to Solutions and Platform -> Developer Services -> Functions.
- Select an existing application or click Create Application.
- Create a new OCI function within your application. See the Oracle Overview of Functions for more details.
- It is recommended to create a boilerplate Python function first and replace the auto generated files with Datadog’s source code:
- Replace
func.py with code from the Datadog OCI repo. - Replace
func.yaml with code from the Datadog OCI repo. DATADOG_TOKEN and DATADOG_HOST must be replaced with your Datadog API key and region logs intake link. - Replace
requirements.txt with code from the Datadog OCI repo.
OCI event
- In the OCI portal, navigate to Solutions and Platform -> Application Integration -> Event Service.
- Click Create Rule to be directed to the Create Rule page.
- Give your event rule a name and description.
- Set your condition as Event Type, service name as Object Storage, and event type as Object - Create.
- Set your action type as Functions.
- Ensure that your function compartment is the same selection you made for OCI Log, OCI Bucket, and OCI Function.
- Select your function application and function (according to the previous installation step.)
- Click Create Rule, then your rule is created and available in the rules list.
For more information on OCI Object Storage, see Getting Started with Events.
Data Collected
Metrics
oci.apigateway.backend_http_responses
(count) | Count of the HTTP responses returned by the back-end services.
Shown as response |
oci.apigateway.bytes_received
(count) | Number of bytes received by the API gateway from API clients.
Shown as byte |
oci.apigateway.bytes_sent
(count) | Number of bytes sent by the API gateway to API clients.
Shown as byte |
oci.apigateway.http_requests
(count) | Number of incoming API client requests to the API gateway.
Shown as request |
oci.apigateway.http_responses
(count) | Number of HTTP responses that the API gateway has sent back.
Shown as response |
oci.apigateway.integration_latency
(gauge) | Time between the API gateway sending a request to the back-end service and receiving a response from the back-end service.
Shown as second |
oci.apigateway.internal_latency
(gauge) | Time spent internally in the API gateway to process the request.
Shown as second |
oci.apigateway.latency
(gauge) | Average time that it takes for a request to be processed and its response to be sent. This is calculated from the time the API gateway receives the first byte of an HTTP request to the time when the response send operation is completed.
Shown as second |
oci.apigateway.response_cache_action
(gauge) | The action taken by the response cache. |
oci.apigateway.response_cache_availability
(gauge) | Availability of the response cache as seen by the API gateway. |
oci.apigateway.response_cache_latency
(gauge) | Total time taken for connect, read, and store operations on the response cache.
Shown as millisecond |
oci.apigateway.subscriber_quota_proportion_used
(gauge) | Proportion of an entitlement's quota that has been consumed by a subscriber. Emitted per request. Calculated as: / .
Shown as fraction |
oci.apigateway.subscriber_rate_limit_proportion_used
(gauge) | Proportion of an entitlement's rate limit that has been consumed by a subscriber. Emitted per request. Calculated as: / .
Shown as fraction |
oci.apigateway.subscriber_requests
(count) | Number of requests made by a subscriber. Emitted per request.
Shown as request |
oci.apigateway.usage_plan_requests
(count) | Number of requests to a given entitlement. Emitted per request.
Shown as request |
oci.oracle_appmgmt.active_requests_by_application
(gauge) | Number of executions active grouped by category.
Shown as request |
oci.oracle_appmgmt.active_user_sessions
(gauge) | Current Active User Sessions by username.
Shown as session |
oci.oracle_appmgmt.active_user_sessions_by_responsibility
(gauge) | Current Active User Sessions grouped by responsibility.
Shown as session |
oci.oracle_appmgmt.capacity_utilization_of_concurrent_managers
(gauge) | Utilized capacity of the concurrent manager.
Shown as percent |
oci.oracle_appmgmt.completed_requests_by_application
(gauge) | Percentage of executions completed grouped by category.
Shown as percent |
oci.oracle_appmgmt.concurrent_processing_component_status
(gauge) | Status of the component. Values are: 1 = Up 0 = Down.
Shown as resource |
oci.oracle_appmgmt.concurrent_requests_by_status
(count) | Concurrent requests by status.
Shown as request |
oci.oracle_appmgmt.deferred_records
(count) | Deferred records grouped by status.
Shown as record |
oci.oracle_appmgmt.executed_programs_by_running_time
(gauge) | Running time of each execution of the program (raw data).
Shown as millisecond |
oci.oracle_appmgmt.forms_database_sessions_per_application
(count) | Number Of Forms Sessions.
Shown as session |
oci.oracle_appmgmt.forms_database_sessions_per_user
(count) | Number Of Forms Sessions.
Shown as session |
oci.oracle_appmgmt.hourly_completed_concurrent_requests_rate
(gauge) | Concurrent Requests Completed by category.
Shown as percent |
oci.oracle_appmgmt.inbound_notifications
(count) | Inbound records grouped by status.
Shown as record |
oci.oracle_appmgmt.internal_concurrent_manager_status
(gauge) | Status of the resource. Values are: 1 = Up 0 = Down.
Shown as resource |
oci.oracle_appmgmt.long_active_concurrent_requests
(gauge) | For pending requests, pending time. For running requests, running time.
Shown as millisecond |
oci.oracle_appmgmt.monitoring_status
(gauge) | Status of the resource. Values are: 1 = Up 0 = Down.
Shown as resource |
oci.oracle_appmgmt.outbound_notifications
(count) | Outbound records grouped by status.
Shown as record |
oci.oracle_appmgmt.queue_details
(count) | Requests grouped by status.
Shown as request |
oci.oracle_appmgmt.users_with_most_pending_requests
(gauge) | Number of requests.
Shown as user |
oci.oracle_appmgmt.users_with_most_running_requests
(gauge) | Number of requests.
Shown as user |
Service Checks
The OCI integration does not include any service checks.
Events
The OCI integration does not include any events.
Troubleshooting
Need help? Contact Datadog support.
Further Reading
Additional helpful documentation, links, and articles:
