This Python script automates the process of updating log retention policies in AWS CloudWatch. It iterates through your log groups and sets a specified retention period for those without an existing policy.

Python Script to Update Log Retention

#!/usr/bin/env python3

import boto3
import time 

aws_profile = 'default'
aws_region = 'eu-west-1'
retention_in_days_number = 5

cwlogs = boto3.Session(profile_name=aws_profile, region_name=aws_region).client('logs')

def list_log_groups(limit_number):
    response = cwlogs.describe_log_groups(limit=limit_number)
    return response['logGroups']

def update_retention(log_group_name, retention_in_days):
    response = cwlogs.put_retention_policy(
        logGroupName=log_group_name, 
        retentionInDays=retention_in_days
    )
    return response

paginator = cwlogs.get_paginator('describe_log_groups')

updated_log_groups = 0

for response in paginator.paginate():
    for log_group in response['logGroups']:
        if 'retentionInDays' not in log_group.keys():
            print("X {loggroup} has no retention policy set, and setting to 5 days".format(loggroup=log_group['logGroupName']))
            update_response = update_retention(log_group['logGroupName'], retention_in_days_number)
            print("RequestID: {rid}, StatusCode: {sc}".format(rid=update_response['ResponseMetadata']['RequestId'], sc=update_response['ResponseMetadata']['HTTPStatusCode']))
            updated_log_groups += 1
            time.sleep(1)
    
print("Updated {count} log groups to the retention on {num} days".format(count=updated_log_groups, num=retention_in_days_number))
            

How to Use the AWS Log Retention Script

To use this script, you'll need to have the AWS CLI configured with the necessary credentials and the boto3 library installed. Modify the aws_profile and aws_region variables to match your AWS environment. The retention_in_days_number variable sets the default retention period in days.

Prerequisites

• AWS CLI configured with appropriate credentials
• boto3 Python library installed (pip install boto3)

Running the Script

1. Save the script to a file, e.g., update_retention.py.
2. Run the script from your terminal: python update_retention.py

Understanding CloudWatch Log Retention

CloudWatch Logs retention policies determine how long your log data is stored. Setting appropriate retention periods helps manage costs and ensures compliance with data retention requirements. For more information, refer to the AWS CloudWatch Logs documentation.

This script uses the boto3 library to interact with the CloudWatch Logs API. It specifically uses the describe_log_groups and put_retention_policy methods.

Additional Resources

• Amazon CloudWatch Official Page
• Configuring the AWS CLI