Libcloudforensics and Cloud Logs

By

In an earlier blog post we introduced Libcloudforensics, a multi-cloud forensic library that can be used to make disk copies and set up virtual machines on GCP and AWS for analysis. We now have added functionality that allows you to query logs from AWS and GCP to aid in your investigations. This blogpost describes how to do that, using the library and accompanying CLI tool.

Cloud logging


Cloud logs are one of the relevant artefacts when investigating cloud incidents. Cloud logs not only provide you with a central place to query your virtual machines logs but also provide audit logs of every (administrative) action taken in the cloud management environment. 


A full set of cloud management and resource logs can quickly show you a timeline of:

  • the actors activity in the actual cloud resources, for example, when an actor started a cronjob in a virtual machine.

  • the cloud project activities, for example, where an actor creates 15 extra virtual machines or adds an admin scoped cloud project user.


Cloud providers each have their own log infrastructure, query (filter) logic and APIs. GCP’s Cloud Logging is their one stop shop for all GCP logs. AWS has multiple cloud logging offerings, namely CloudTrail (for AWS account activity logs) and CloudWatch (for application logs) contain information that is useful for forensic investigations. 


GCP Cloud logging and AWS CloudTrail are currently supported by libcloudforensics. Libcloudforensics provides you with an API and command line interface (CLI) tool to easily integrate, query and export these logs for further analysis. AWS CloudWatch will be supported in the future. 

GCP Cloud Logging

GCP offers a rich query language to filter and fetch all logs in a cloud project. The query language is defined here and can be used within libcloudforensics.


As a simple example, let’s query what kind of activity we see in our virtual machine logs that match a particular indicator (in this case the 'bad' IP address 1.2.3.4).

from libcloudforensics.providers.gcp.internal import log as gcp_log

logs = gcp_log.GoogleCloudLog(project_id='my-gcp-project')

results = logs.ExecuteQuery(qfilter='resource.type="gce_instance" AND

    "1.2.3.4"')


We can also easily do this with the libcloudforensics CLI tool.

$ cloudforensics gcp my-gcp-project querylogs --filter    

    'resource.type="gce_instance" AND "1.2.3.4"'

AWS CloudTrail

AWS CloudTrail does not not contain virtual machine logs but does contain the logs that show what project actions have been performed. Examples are the creation or alteration of AWS services, AWS console activity and API logins and actions. CloudTrail logs are very useful in investigating malicious AWS project activity.


CloudTrail filtering is limited as AWS only supports one key-value pair per filter. See the CloudTrail event documentation for supported key values.


As an example, let’s see if any access keys were created. Access keys can be used to access the AWS services and depending on their scope can be used to control all resources. 

from libcloudforensics.providers.aws.internal import account
from libcloudforensics.providers.aws.internal import log as aws_log
 
ct = aws_log.AWSCloudTrail(account.AWSAccount(
                          default_availability_zone='eu-central-1a'))
result = ct.LookupEvents('EventName,CreateAccessKey')

We can perform the same query using the libcloudforensics CLI tool including a start and end time in the filter.

$ cloudforensics aws eu-central-1a querylogs --start '2020-05-12 11:30:00' --end '2020-05-13 21:00:00' --filter 'EventName,CreateAccessKey'

Conclusion

We hope this blog post has given you inspiration on how to further automate your cloud incident response. If you have questions, reach out on the Open Source DFIR Slack, you can find us in the  libcloudforensics channel.

Resources

Comments

Post a Comment

Popular posts from this blog

Parsing the $MFT NTFS metadata file

By
Background Numerous articles have been written about parsing the $MFT NTFS metadata file before. Some of these explain the technique itself [ 1 ] and others mostly focus on how to use tools. Rarely do articles mention the pros and cons of the technique from a digital forensics perspective [ 2 ].  What is $MFT parsing?  For readers not familiar with the NTFS file system, MFT stands for Master File Table. This table is stored in a file system metadata file, meaning that as far as the file system is concerned it is a file, however that the file contains file system metadata not “regular” content. The name of this file is “$MFT”.  The MFT consists of a sequence of (predetermined) fixed-size entries (or MFT entries). These entries are typically 1024 bytes in size, however the size is defined in the NTFS volume header and in a “regular” MFT entry itself.  The first 4-bytes of a “regular” MFT-entry (or record) starts with the signature “FILE”. The MFT entry can...
Read more

Incident Response in the Cloud

By
Many organizations have begun moving a majority of their services towards the cloud in recent years. As a result, attackers have shifted their focus towards the cloud. This has resulted in new techniques and methods specifically designed to compromise cloud infrastructure, like the recent SALTSTACK vulnerability that was widely exploited [ 1 ]. Therefore it is critical for these organizations to have an incident response team that understands the new risks attached to cloud and how cloud can make incident response easier or harder.  In this blog post we will walk you through each phase you may encounter in traditional incident response and highlight the differences when adopting cloud computing. It is aimed towards both those who are new to incident response and cloud computing. We’ve included insights that will benefit organizations taking that step towards cloud who want to ensure they are prepared to respond efficiently to cloud incidents.  Traditional Incident Response Be...
Read more

Container Forensics with Docker Explorer

By Anonymous
  Introduction As previous blog posts on cloud forensics have noted, applications are increasingly being deployed using container orchestration frameworks such as Kubernetes, especially in cloud environments.  Similar to traditional deployments on physical servers or virtual machines (VMs), when a containerised application has a security issue it can lead to a compromise of the underlying compute architecture. In the case of container deployments this means a compromise of the container itself, the container host, or even a wider cluster compromise via abuse of orchestration tools. Often digital forensics is required to establish what went wrong and remediate any issues. This article will provide an introduction to container forensics with Docker Explorer by working through a scenario involving a compromised container running within a Kubernetes cluster. Although Kubernetes is briefly mentioned, this article will focus on analysis of an individual container rather than the wi...
Read more