Amazon Web Services (AWS) provides various tools which you can use to monitor AWS WAF (Web Application Firewall).
In our previous blog, we have seen how we can centrally configure and manage AWS WAF rules across multiple accounts and applications using AWS Firewall Manager.
AWS now provides a new service for detailed logging called Amazon Kinesis Data Firehose, which is a fully managed service for delivering real-time streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon Elasticsearch Service (Amazon ES), and Splunk.
Amazon Kinesis Data Firehose stores all logs in the S3 bucket. These detailed logs give us more information about why certain rules are triggered and why certain rules are blocked with our specific ACL (Access Control List) rules.
For Amazon S3 destinations, streaming data is delivered to your S3 bucket. If data transformation is enabled, you can optionally back up source data to another Amazon S3 bucket.This is done in two steps assuming WAF is already implemented (if not, first implement WAF with the help of our blog):
- Adding Kinesis Data Firehose stream and choosing the destination for the data from Amazon S3, Amazon Elasticsearch, and Amazon Redshift.
- From existing WAF configurations, and enabling detailed logs using Firehose instance.
Implementation
Follow these steps for implementing Amazon Kinetic Data Firehose:
- Select ‘Kinesis Data Firehose’ for a new instance.
- Add a unique name for Firehose
- Choose a source to send records to the delivery stream
- Transform source records
- Convert record format
- Select the destination
- Configure all setting as per our requirement for the S3 bucket and select ‘Enabled’
- Create IAM role with the following policy :
With the name firehose_delivery_role
- iam:CreateServiceLinkedRole
- firehose:ListDeliveryStreams
- firehose:PutLoggingConfiguration
Select created IAM role firehose_delivery_role in the Kinesis Data Firehose instance creation
- Review all configurations and create a delivery stream
Finally, you will able to see the active firehose in the console.
- Enable logging
Navigate to the WAF console, choose the region where the WAF is configured in the Logging tab and configure the section for ‘Enable Logging’.We should now be seeing detail logging in Kinesis service:For testing our setup, we can use demo dataThat’s it. Have you already configured and started using Amazon Kinesis Data Firehose? What challenges did you face? What did you learn? If you need help, please leave a comment below and an AWS expert will get in touch with you.
