Overview

This guide walks you through how to redact all emails, except the ones from a specific email domain (for example, @test.com), in your logs.

Set up a grok parser in your logs pipeline

If the email domain you do not want redacted is not an existing log attribute, set up a grok parser to identify all logs with the email domain and add it as an attribute.

1. Navigate to Log Pipeline.
2. Select your pipeline.
3. Click Add processor.
4. Select Grok Parser.
5. Enter a name for the grok parser.
6. Define the parsing rules to identify all logs with the email address. For example, if these are the log messages that contain email addresses with the domain:

   message successfully sent to 123@test.com
   

   message successfully received from 256@test.com
   

   Then use the following parsing rules:

   MyParsingRule1 message successfully sent to %{notSpace:user_handle}@%{notSpace:domain}
   
   MyParsingRule2 message successfully received from %{notSpace:user_handle}@%{notSpace:domain}
   

   Note: You don’t need to keep the username. For example, if you want to redact all emails with the domain test.com, then for an email like hello@test.com, discard the username hello and just keep the domain test.com.
7. Click Save.

Navigate to Log Explorer to confirm that new logs coming in with those emails are getting processed as expected.

Add the email domain attribute as a facet

1. In Log Explorer, select a log that contains an email with the specified domain.
2. Click on the cog next to the domain attribute you just created.
3. Select Create facet for….
4. Optionally, add the facet to a group in the Advanced Options section.
5. Click Add.

Configure the Sensitive Data Scanner scanning group to filter out logs with your domain attribute

Update your Sensitive Data Scanner’s scanning group to filter out logs with the domain attribute that you created, so only logs that do not have that email domain are redacted.

1. Navigate to the Sensitive Data Scanner Configuration page.
2. Click the pencil icon to the left of the scanning group you want to update.
3. In the Filter field, add the domain attribute so that logs with that attribute are filtered out. For example, to filter out logs with the email domain test.com, add -@domain:test.com to the filter query.

4. Click Update.

Navigate to Log Explorer to confirm that the new logs coming in do not have emails with the specified domain redacted.

Further reading

Additional helpful documentation, links, and articles:

Set up Sensitive Data ScannerDOCUMENTATION