Bringing an End to Sorrow.... New Plaso Release

By Anonymous
Barren fields will bear again, plaso’s return with version 1.3 brings an end to sorrow … log2timeline and plaso will live on with a brand new release of plaso that you can enjoy in between hanging out at the pool, surfing or just lying on the beach while reciting old Nordic poems.

Version 1.3 the Eir Edition Released
The plaso development team is happy to announce the release of plaso, version 1.3 the Eir Edition. Eir somewhat appropriate being the "god" associated with medical skill, part of our new release naming schema of Norse gods and goddesses.

With this brand new version we bring you improved stability, bug fixes and major refactors and various other glorious features people are getting comfortably used to with each new release.

Now answer me, Fjolsvith, the question I ask, for now the truth would I know. What are those new shiny features being introduced at this time?

What has changed since Griswold?

We've noticed a lot of stability issues in version 1.2 (aka Griswold) regarding worker processes not responding, hence our focus for 1.3 has been fixing the more serious issues and improving code health. To make a long story short we've refactored many parts of the code base to fix or work-around some underlying issues and to make plaso easier to maintain. Since some of these issues do not originate within the plaso codebase and not all of them are easy to fix we'll continuing bug fixing and improving code health after this release.

New hotness

Enough on the health issues, let's talk about the new hotness in version 1.3.

Changes to the core

  • signature-based parser pre-filtering was added; this means that the parser selection first scans for parsers that define a format signature (magic number). If there are matches only the relevant parsers are applied, if not only the remaining parsers that don't define a format signature are applied.
  • initial version of file content hashing support; plaso now supports calculating SHA-256, SHA-1 and MD5 digest hashes of file content.
  • Windows Event Log event messages support
  • improvements to compressed and archive file support; this is something that is still work in progress, but as a first step gzip (.gz) and bzip2 (.bz2) compressed files are now decompressed by default. For those interested you can follow progress on issue 230.
  • BitLocker encrypted (BDE) volumes; this was somewhat available in dfVFS since plaso version 1.2.0 but now has been added to plaso as well. Note that there are still some rough edges that need to be polished, for more details see issue 268.
If you are developing for plaso know that dfVFS now uses a stricter caching strategy and requires open file handles to be explicitly closed. Other changes include but are not limited to; parser name now being a parser chain to adequately describe what parsers were used to extract the event, changed the parser context into parser mediator, split off build and update dependency utility scripts to l2tdevtools, and various other refactors. Now that 1.3 is out we'll be updating the codelabs with much more details on this.

Changes to parsers and parser plugins

New additions:
  • Chrome preferences (with thanks to Eric John)
  • Restore points log file (rp.log)
  • Safari Binary cookie
  • Portable Executable (PE) file
  • ESE database file formats
    • Windows 8 File History
  • Windows Registry plugins
    • WinReg Timezone Plugin (with thanks to Francesco)
Improvements:
  • MSIECF parser added cache directory name support (issue 135)
  • ESE database parser added long value support
  • AppCompatCache Registry plugin added Windows 10 support

Changes to output formats

We have done a large overhaul of this part of the code base but managed to squeeze in some functional changes as well:
  • integration with the current Timesketch release
  • JSON output with a single line per event
  • changes to TLN to add non-ambiguous time indication (issue 5)
  • 'null' plugin, which produces no output (useful for testing, and using analysis plugins that do tagging)
With thanks to Jason Blanks for adding tests for the 4n6time output modules.

    Changes to analysis plugins

    Eir sees some changes and additions to Plaso's analysis plugins. Several make use of the new hashing support in log2timeline. Specific new analysis plugins are:
    • file_hashes: Lists all the hashes for all files processed by Plaso.
    • tagging: This plugin incorporates the rule-based tagging functionality from the plasm tool, which has now been deprecated.
    • virustotal: Looks up PE files using the VirusTotal API, and tags events derived from the files with useful tags. One thing to note - this plugin may take a (very) long time, as VirusTotal rate-limits the number of requests made with a free API key.
    • viper: Looks up PE files in a Viper instance, and annotates events derived from the files. Handy for spotting known pieces of malware that you're tracking.

    Changes to tools

    The tools have been moved out of "plaso/frontend" to the "tools" directory. Besides the move we also separated the tool and front-end functionality; working towards to reuse functionality of plaso in tools like e.g. Timesketch.

    Regarding the tools themselves:
    • image_export and preg were improved
    • we said goodbye to pprof, pshell and plasm and removed them
    • JSON serialization is now the default for the storage, which has significantly improved serialization speed
    • log2timeline now comes with a first iteration of multi volume support (with thanks to the DC3 team).
    An experimental feature worth mentioning is the top-like status view for log2timeline:
    log2timeline.py --status-view=window storage.plaso image.raw

    Changes to documentation

    We are in the process of moving the documentation from Google Sites to GitHub Wiki mirrored on readthedocs. The API documentation now uses sphinx-doc. If you're just after the 1.3 documentation, you can see it here.

    Changes to deployment

    Last but not least are the changes to deployment. We heard some of you grunting about the number of dependencies and the challenges of building some of them. Not much we can do there, what we are trying to do instead is making it easier to keep up to date with them. Hence we started l2tdevtools, l2tbinaries and GIFT.

    What are we planning next

    For plaso 1.4 we will continue refactoring and bug fixing. Besides that we are also planning new and shiny features:
    • integration of artifacts for preprocessing and presets
    • more parsers, parser plugins, analysis plugins
    • storage refactor
    • improved NTFS support (ADS, $UsnJrnl, $LogFile)
    • improved multi volume support (LVM, LUKS)
    • split of Windows Registry handling to a separate project
    Development workflow changes to use Travis and AppVeyor testing in the codereview process.

    Where to get plaso 1.3?

    See plaso's users guide.

    Comments

    Post a Comment

    Popular posts from this blog

    Parsing the $MFT NTFS metadata file

    By
    Background Numerous articles have been written about parsing the $MFT NTFS metadata file before. Some of these explain the technique itself [ 1 ] and others mostly focus on how to use tools. Rarely do articles mention the pros and cons of the technique from a digital forensics perspective [ 2 ].  What is $MFT parsing?  For readers not familiar with the NTFS file system, MFT stands for Master File Table. This table is stored in a file system metadata file, meaning that as far as the file system is concerned it is a file, however that the file contains file system metadata not “regular” content. The name of this file is “$MFT”.  The MFT consists of a sequence of (predetermined) fixed-size entries (or MFT entries). These entries are typically 1024 bytes in size, however the size is defined in the NTFS volume header and in a “regular” MFT entry itself.  The first 4-bytes of a “regular” MFT-entry (or record) starts with the signature “FILE”. The MFT entry can...
    Read more

    Incident Response in the Cloud

    By
    Many organizations have begun moving a majority of their services towards the cloud in recent years. As a result, attackers have shifted their focus towards the cloud. This has resulted in new techniques and methods specifically designed to compromise cloud infrastructure, like the recent SALTSTACK vulnerability that was widely exploited [ 1 ]. Therefore it is critical for these organizations to have an incident response team that understands the new risks attached to cloud and how cloud can make incident response easier or harder.  In this blog post we will walk you through each phase you may encounter in traditional incident response and highlight the differences when adopting cloud computing. It is aimed towards both those who are new to incident response and cloud computing. We’ve included insights that will benefit organizations taking that step towards cloud who want to ensure they are prepared to respond efficiently to cloud incidents.  Traditional Incident Response Be...
    Read more

    Container Forensics with Docker Explorer

    By Anonymous
      Introduction As previous blog posts on cloud forensics have noted, applications are increasingly being deployed using container orchestration frameworks such as Kubernetes, especially in cloud environments.  Similar to traditional deployments on physical servers or virtual machines (VMs), when a containerised application has a security issue it can lead to a compromise of the underlying compute architecture. In the case of container deployments this means a compromise of the container itself, the container host, or even a wider cluster compromise via abuse of orchestration tools. Often digital forensics is required to establish what went wrong and remediate any issues. This article will provide an introduction to container forensics with Docker Explorer by working through a scenario involving a compromised container running within a Kubernetes cluster. Although Kubernetes is briefly mentioned, this article will focus on analysis of an individual container rather than the wi...
    Read more