Introducing OSDFIR Infrastructure: Automating Deployment and Integration of Open Source DFIR Tools to Kubernetes

By

Overview

As digital threats continue to grow, organizations need to be able to respond quickly and effectively to security incidents. One critical component of incident response is having the right set of tools at hand to analyze and respond to threats. However, manually deploying and integrating multiple open source DFIR tools can be a time-consuming and error-prone process, causing significant delays in incident response times, which can lead to a higher risk of damage to the organization.

To address this challenge, we are excited to share an open source repository for deploying and managing Open Source Digital Forensics & Incident Response (OSDFIR) tools in Kubernetes. OSDFIR Infrastructure automates the tedious deployment and configuration steps, removing the manual labor involved in setting up and maintaining an enterprise DFIR Infrastructure in Kubernetes. 

Using this has helped save hours of time by reducing the time required to deploy, configure, and maintain each tool to a matter of minutes and we would love to get your feedback on whether it works for you too.

The Motivation for OSDFIR Infrastructure


To evaluate our deployment capabilities, we tested the time it took to deploy and configure Timesketch, Turbinia, and dfTimewolf, to work together. Our experience showed a few challenges that can be summarized as:

  • Time Consuming

    • Took several hours of time for each tool deployed

    • Port-forwarding from VMs in cloud can be unreliable

  • Unclear Documentation

    • No clear documentation around how these tools work together

    • No clear documentation on setting them up to work together

  • Difficult to Maintain

    • Maintenance costs grows for each tool added

    • Incompatibility issues with tool upgrades due to limited integration testing

  • Difficult to Scale

    • Scaling Docker containers requires a lot of work

    • No shared file storage, such as NFS, data is stored in each VM

Given these challenges, there is a need for a solution that streamlines the deployment, integration, and scaling of multiple open-source DFIR tools, reducing the time and effort required to set up, maintain, and scale an effective incident response infrastructure.

Introducing OSDFIR Infrastructure

OSDFIR Infrastructure is an open-source repository that provides a set of Helm charts and simplified instructions for automating the deployment and integration of multiple open-source DFIR tools in Kubernetes. 

The repository has the potential to support the deployment of several popular open-source DFIR tools, including:

  • Timesketch - for collaborative forensic timeline analysis. Using sketches you and your collaborators can organize and work together while using analyzers to help identify patterns and trends in data.

  • Turbinia - for automating processing of forensic evidence at scale and helps perform analysis to find some of the most prevalent badness, reducing response time.

  • DFTimewolf - a framework for orchestrating forensic collection, processing and data export, helping data be passed along between tools.

  • Libcloudforensics - a library to help acquire forensic evidence from cloud platforms. 

  • Plaso (and related projects such as dfVFS, libyal) - used to extract and parse data from a variety of sources (e.g. files, logs, disks, Registry hives) then output them into a correlated super timeline.

  • Hashr - allows you to generate your own hash sets by extracting and hashing the actual files from complex data sources  (e.g. raw disk, GCP disk, Windows WSUS update packages, ISO file, tar file).

  • DFDewey - a string extraction, indexing, and searching tool that can collect strings from files and raw disks then index them to be searchable.

  • GRR - an incident response framework focused on remote live forensics containing an agent to be installed on target systems and a server infrastructure that can manage and talk to the clients.


The Benefits of OSDFIR Infrastructure


The benefits of OSDFIR Infrastructure can be summarized into the diagram below, where current processes require a lot of steps versus a single simplified deployment through OSDFIR Infrastructure.


The benefits can be further explained by the following key points:

Faster Deployment Times

Deployment and integration now takes an hour or less, or even minutes of time with a pre-existing Kubernetes cluster, rather than hours to days of time individually deploying and configuring each tool. This allows responders to focus on more critical tasks, improving overall incident response times.

Provides Consistent Configuration

Automating the deployment process in OSDFIR Infrastructure helps responders ensure that configurations are consistent and secure by default, reducing the risk of misconfiguration or human error.

Certified Deployments

OSDFIR Infrastructure provides the community with a quick and easy way to set up multiple DFIR tools that each play a unique role in incident response. This eliminates the need to search through multiple tool documentation for compatibility.

A shared infrastructure allows for the sharing of other resources, such as storage and required third-party dependencies such as Redis. This reduces the complexity of the infrastructure by eliminating the need to manage redundant resources.

Easy to Scale

OSDFIR Infrastructure can be installed with minimal resources, such as Minikube, and then upgraded with a single command. Anything from the persistent volume size to worker resources, to deploying a load balancer for external connectivity and many more, can be individually upgraded, and the rollout of the change automatically updates on the infrastructure causing little to no downtime.

With Kubernetes, you can also automatically scale resources based on defined metrics such as CPU utilization. This has already shown its benefits with Turbinia, where the number of workers automatically scales based on processing demand.

Improves Reliability

With a consistent way to deploy DFIR tools, we can run more extensive cross tool integration tests that can help identify issues between them. Helm also provides built-in Chart linting and testing, making it possible to run more automated tests, overall improving the reliability of DFIR tools.

Kubernetes also makes it easy to port-forward applications, allowing responders a fast and secure way to access the infrastructure locally.

Getting Started

We are excited to announce that our first release of OSDFIR Infrastructure is now available to use, integrating Turbinia, Timesketch, and dfTimewolf. To get started, please visit our getting started guide for setting up your own infrastructure locally on your machine using Minikube.

In summary

Manually deploying and integrating, maintaining, and scaling multiple DFIR tools can be a time-consuming and error-prone process. With that came a need for a solution that reduces the time and effort required to set up and maintain an effective incident response infrastructure.

OSDFIR Infrastructure helps to fill that gap by streamlining the deployment, integration, maintenance, and scaling of multiple open-source DFIR tools. It provides the following key benefits:

  • Faster Deployment Times: Deployment can take a few minutes rather than hours to days allowing responders to focus on more critical tasks.

  • Provides Consistent Configuration: Integration comes automatically, reducing risk of misconfiguration or human error.

  • Certified Deployments: No more looking at multiple tools documentation, reduces complexity from shared resources, centralized way to learn about many DFIR tools.

  • Easy to Scale: Can be installed on minimal resources and any component can be modularly scaled with a single command. Kubernetes autoscaling is a powerful feature.

  • Improves Reliability: Built-in Chart linting and testing. More ways to run integration tests to catch issues between them. Port-forwarding locally running applications is easy and reliable.

We hope this project serves as an accessible way for anyone to get started with open-source DFIR tools without spending hours of time setting them up to work together. The easiest way to stay up-to-date is by starring our repository or any of the tools mentioned above for tool-specific updates.  We would love to hear your feedback on how this works for you!

Comments

Post a Comment

Popular posts from this blog

Parsing the $MFT NTFS metadata file

By
Background Numerous articles have been written about parsing the $MFT NTFS metadata file before. Some of these explain the technique itself [ 1 ] and others mostly focus on how to use tools. Rarely do articles mention the pros and cons of the technique from a digital forensics perspective [ 2 ].  What is $MFT parsing?  For readers not familiar with the NTFS file system, MFT stands for Master File Table. This table is stored in a file system metadata file, meaning that as far as the file system is concerned it is a file, however that the file contains file system metadata not “regular” content. The name of this file is “$MFT”.  The MFT consists of a sequence of (predetermined) fixed-size entries (or MFT entries). These entries are typically 1024 bytes in size, however the size is defined in the NTFS volume header and in a “regular” MFT entry itself.  The first 4-bytes of a “regular” MFT-entry (or record) starts with the signature “FILE”. The MFT entry can also be filled wit
Read more

Incident Response in the Cloud

By
Many organizations have begun moving a majority of their services towards the cloud in recent years. As a result, attackers have shifted their focus towards the cloud. This has resulted in new techniques and methods specifically designed to compromise cloud infrastructure, like the recent SALTSTACK vulnerability that was widely exploited [ 1 ]. Therefore it is critical for these organizations to have an incident response team that understands the new risks attached to cloud and how cloud can make incident response easier or harder.  In this blog post we will walk you through each phase you may encounter in traditional incident response and highlight the differences when adopting cloud computing. It is aimed towards both those who are new to incident response and cloud computing. We’ve included insights that will benefit organizations taking that step towards cloud who want to ensure they are prepared to respond efficiently to cloud incidents.  Traditional Incident Response Before jump
Read more

Container Forensics with Docker Explorer

By Anonymous
  Introduction As previous blog posts on cloud forensics have noted, applications are increasingly being deployed using container orchestration frameworks such as Kubernetes, especially in cloud environments.  Similar to traditional deployments on physical servers or virtual machines (VMs), when a containerised application has a security issue it can lead to a compromise of the underlying compute architecture. In the case of container deployments this means a compromise of the container itself, the container host, or even a wider cluster compromise via abuse of orchestration tools. Often digital forensics is required to establish what went wrong and remediate any issues. This article will provide an introduction to container forensics with Docker Explorer by working through a scenario involving a compromised container running within a Kubernetes cluster. Although Kubernetes is briefly mentioned, this article will focus on analysis of an individual container rather than the wider clust
Read more