Introducing Libcloudforensics

By


Blog co-authored by Theo Giovanna and hacktobeer.

An increasing number of businesses are migrating their core IT environment to the cloud. Cloud computing offers these businesses a flexible and dynamic IT infrastructure, with on-demand availability of computing resources. Due to on-demand scaling cloud computing is also an interesting target for abuse. For people working in Digital Forensics and Incident Response (DFIR) this can mean you have to deal with a surge of security incidents at once [1].

Setting up an investigation-ready environment and responding to cloud incidents can be time-consuming. Already short-staffed DFIR teams need an efficient way to do this in order to keep up. This blog post introduces libcloudforensics, a library that is multi-cloud provider compatible and that provides a unified API that helps in reducing the time to set-up an investigation environment and automating part of responding to cloud compromises.

Cloud Forensics

For a more detailed explanation of cloud forensics and the distinction between performing digital forensics on the cloud and in the cloud, see this blog post about conducting digital forensics at scale. With libcloudforensics we aim to facilitate both. 

On the Cloud

Core features of libcloudforensics include:
  • the ability to produce forensic copies of disks hosted in the cloud;
  • the ability to query cloud logs to aid in investigations;
  • automating the creation of analysis virtual machines to conduct an investigation;
  • easy to use CLI tools.
This post is part of a series of three. This first one will present an overview of the disk copying functionality.

Making disk copies in the cloud

In a non-cloud investigation it is common practice to make a bit-by-bit replica of the physical source disk (forensic copy). In the cloud, we do not have physical disks and creating a forensic copy from a disk usually involves the following steps:
  1. Creating a snapshot of the disk.
  2. Making a new disk from the snapshot. Depending on the cloud provider and the requirements, this step might involve extra sub-steps (e.g. if resources need to be shared with an account that is different from the source account).
  3. Cleaning up intermediate resources.

Digital preservation in the cloud represents challenges: users have limited control over data content, format, associated metadata, and more generally of the execution environment. Moreover, cloud services offering are subject to change without previous warning, adding challenges to the preservation of digital evidence in the cloud [2].

Additionally, cloud instances (virtual machines) may have dozens of disks and typically all are relevant for the investigation. Manually creating forensic copies out of these disks through the cloud provider’s user interface is not scalable.

An additional complicating factor is that different cloud providers have different user interfaces and dealing with those during an investigation is cumbersome. Libcloudforensics addresses this problem by exposing a single API which will take care of these steps without any overhead. It also offers possibilities to carry out common lower-level operations useful when doing incident response, like listing and retrieving instances, disks, and more.

Setting up analysis VMs in the Cloud

Libcloudforensics allows you to automate the creation of analysis virtual machines with a predefined set of tools and configuration to conduct the investigation. Libcloudforensics allows you to:
  • Customize the predefined set of tools that suits your needs;
  • Attach disk copies created with the library, thus speeding up the process of getting your environment ready.
Libcloudforensics gives you the ability to customize the VM’s image and its processing power, thus fully making use of available cloud computing capabilities. It currently uses a startup script, giving users a flexible way to install custom tools. By default, the library installs popular open source DF tools such as Plaso and TSK from the GIFT PPA.

Roadmap

Currently most cloud providers add new features every day. This means that libcloudforensics will continue to adapt to those changes. The next paragraphs discuss some changes that we are planning for.

Adding support for more cloud providers (Horizontal)

Libcloudforensics currently supports AWS and GCP. We are looking into supporting Azure in the near future and will continue to work with the forensic community to see which cloud providers we should support. 

Continuous testing to stay up-to-date (Vertical)

The core functionalities of libcloudforensics need to stay up-to-date: both when new cloud API versions are released as well as when new artifacts are identified. This requires rigorous end-to-end tests for the library. 

Contact

If you have questions, reach out on the Open Source DFIR Slack, there is a libcloudforensics channel.

If you want to contribute support or found a bug please open an issue or pull request on GitHub.

Resources

Popular posts from this blog

Parsing the $MFT NTFS metadata file

By
Background Numerous articles have been written about parsing the $MFT NTFS metadata file before. Some of these explain the technique itself [ 1 ] and others mostly focus on how to use tools. Rarely do articles mention the pros and cons of the technique from a digital forensics perspective [ 2 ].  What is $MFT parsing?  For readers not familiar with the NTFS file system, MFT stands for Master File Table. This table is stored in a file system metadata file, meaning that as far as the file system is concerned it is a file, however that the file contains file system metadata not “regular” content. The name of this file is “$MFT”.  The MFT consists of a sequence of (predetermined) fixed-size entries (or MFT entries). These entries are typically 1024 bytes in size, however the size is defined in the NTFS volume header and in a “regular” MFT entry itself.  The first 4-bytes of a “regular” MFT-entry (or record) starts with the signature “FILE”. The MFT entry can also be filled wit
Read more

Incident Response in the Cloud

By
Many organizations have begun moving a majority of their services towards the cloud in recent years. As a result, attackers have shifted their focus towards the cloud. This has resulted in new techniques and methods specifically designed to compromise cloud infrastructure, like the recent SALTSTACK vulnerability that was widely exploited [ 1 ]. Therefore it is critical for these organizations to have an incident response team that understands the new risks attached to cloud and how cloud can make incident response easier or harder.  In this blog post we will walk you through each phase you may encounter in traditional incident response and highlight the differences when adopting cloud computing. It is aimed towards both those who are new to incident response and cloud computing. We’ve included insights that will benefit organizations taking that step towards cloud who want to ensure they are prepared to respond efficiently to cloud incidents.  Traditional Incident Response Before jump
Read more

Container Forensics with Docker Explorer

By Anonymous
  Introduction As previous blog posts on cloud forensics have noted, applications are increasingly being deployed using container orchestration frameworks such as Kubernetes, especially in cloud environments.  Similar to traditional deployments on physical servers or virtual machines (VMs), when a containerised application has a security issue it can lead to a compromise of the underlying compute architecture. In the case of container deployments this means a compromise of the container itself, the container host, or even a wider cluster compromise via abuse of orchestration tools. Often digital forensics is required to establish what went wrong and remediate any issues. This article will provide an introduction to container forensics with Docker Explorer by working through a scenario involving a compromised container running within a Kubernetes cluster. Although Kubernetes is briefly mentioned, this article will focus on analysis of an individual container rather than the wider clust
Read more