The Future of the DigitalCorpora

By

The Future of the DigitalCorpora

By Simson Garfinkel, copied with permission

The DigitalCorpora corpus currently hosts approximately 19TB of data, serving as a foundational, free open data resource for the global digital forensics and security research communities.

After 20 years of maintaining this vital community asset, I am seeking a qualified individual, group, or institution to take over and support DigitalCorpora.

What’s Inside the 19TB Corpus?

DigitalCorpora is a goldmine for researchers, educators, and students. The data is broadly categorized into:

  • Forensics Scenarios & Challenges: Real-world digital forensics scenarios (some with solutions) alongside comprehensive archives of past DFRWS challenges.

  • The SAFEDOCS & UNSAFEDOCS Collection: The complete close-out data from the DARPA SAFEDOCS program. This includes terabytes of PDFs harvested from the public internet, some of which are segregated into the "UNSAFEDOCS" corpus where they await detailed analysis.

  • Network & Storage Artifacts: Extensive packet captures (PCAPs) and raw disk images.

  • Government Web Archives: Just under 1 million files scraped from U.S. Government (USG) websites during the mid-2000s.

Tech Stack

AWS covers data storage for free under its OpenData program.

1. The Website

The public face of the project is currently a WordPress website. To improve security and lower maintenance overhead, I am in the middle of transitioning this to a static site. That should be finished in a month or so.

2. Solution Requests & Auto-Responder

Faculty members use a web form to request passwords for protected forensics solutions.

  • The Reality: The decryption password hasn't been changed in 20 years. Does it need to be changed? I don’t know! Changing it would require re-encrypting the encrypted ZIP and PDF files, a process that could be readily automated

  • The Workflow: The process relies on an auto-responder that fires after I personally review and approve the request. I have considered handing this approval process over to a Large Language Model (LLM), but haven't built it yet.

3. Download Statistics

I previously wrote custom software to track and analyze download statistics. However, bot activity has exploded over the past six months, blowing up the current stats system.

  • The Plan: I designed a scalable infrastructure transition using AWS Lambda and DynamoDB, but I haven't implemented it yet. It’s designed to operate 100% within the free tier of both Lamda and DynamoDB.

Time Commitment & The "Intel" Payoff

Running DigitalCorpora is surprisingly lightweight. I spent less than 10 minutes per week responding to solution requests. However, every now and then, something comes up that takes 30-45 minutes.

The real reward of managing this gatekeeping process is the incredible data intelligence and situational awareness it generates. Reviewing these requests lets you map global adoption, documenting an active footprint of more than 300 faculty members and over 10,000 students worldwide. If the next manager of DigitalCorpora wishes to submit grant proposals, this verified impact data is incredibly powerful for demonstrating community need and securing funding.

Want to make a difference and interested in Taking Over?

DigitalCorpora is a highly respected, high-impact resource that truly moves the needle for cybersecurity education globally. If you have experience with AWS infrastructure, a passion for digital forensics, and the capacity to steward this platform into its next decade, I would love to hear from you via simsong at gmail.com.

Comments

Post a Comment

Popular posts from this blog

Parsing the $MFT NTFS metadata file

By
Background Numerous articles have been written about parsing the $MFT NTFS metadata file before. Some of these explain the technique itself [ 1 ] and others mostly focus on how to use tools. Rarely do articles mention the pros and cons of the technique from a digital forensics perspective [ 2 ].  What is $MFT parsing?  For readers not familiar with the NTFS file system, MFT stands for Master File Table. This table is stored in a file system metadata file, meaning that as far as the file system is concerned it is a file, however that the file contains file system metadata not “regular” content. The name of this file is “$MFT”.  The MFT consists of a sequence of (predetermined) fixed-size entries (or MFT entries). These entries are typically 1024 bytes in size, however the size is defined in the NTFS volume header and in a “regular” MFT entry itself.  The first 4-bytes of a “regular” MFT-entry (or record) starts with the signature “FILE”. The MFT entry can...
Read more

Incident Response in the Cloud

By
Many organizations have begun moving a majority of their services towards the cloud in recent years. As a result, attackers have shifted their focus towards the cloud. This has resulted in new techniques and methods specifically designed to compromise cloud infrastructure, like the recent SALTSTACK vulnerability that was widely exploited [ 1 ]. Therefore it is critical for these organizations to have an incident response team that understands the new risks attached to cloud and how cloud can make incident response easier or harder.  In this blog post we will walk you through each phase you may encounter in traditional incident response and highlight the differences when adopting cloud computing. It is aimed towards both those who are new to incident response and cloud computing. We’ve included insights that will benefit organizations taking that step towards cloud who want to ensure they are prepared to respond efficiently to cloud incidents.  Traditional Incident Response Be...
Read more

Container Forensics with Docker Explorer

By Anonymous
  Introduction As previous blog posts on cloud forensics have noted, applications are increasingly being deployed using container orchestration frameworks such as Kubernetes, especially in cloud environments.  Similar to traditional deployments on physical servers or virtual machines (VMs), when a containerised application has a security issue it can lead to a compromise of the underlying compute architecture. In the case of container deployments this means a compromise of the container itself, the container host, or even a wider cluster compromise via abuse of orchestration tools. Often digital forensics is required to establish what went wrong and remediate any issues. This article will provide an introduction to container forensics with Docker Explorer by working through a scenario involving a compromised container running within a Kubernetes cluster. Although Kubernetes is briefly mentioned, this article will focus on analysis of an individual container rather than the wi...
Read more