What flies there? What fares there? Or moves through the air? Plaso 1.5 - Gná released

By Anonymous

A Vanir asked:
"What flies there?
What fares there?
or moves through the air?"
Gná replied:
"I fly not
though I fare
and move through the air
on Hofvarpnir"
[The Prose Edda, Penguin Classics]

Version 1.5 - Gná released

The Plaso development team is overjoyed to announce the release of Plaso 1.5, codenamed Gná.


As befitting a messenger goddess, Gná brings word of new additions to Plaso, as well as resolving some long-standing challenges to help drive the tool forwards. She rides a flying horse, Hófvarpnir, which while not as cool as a chariot pulled by cats, is still pretty nifty.



As with previous releases, Plaso Gná is packed with new features, changes to the core and improved stability and reliability. Now for the juicy details!

What has changed since Plaso 1.4, Freya?

Gná brings with a bunch of under-the-hood changes. The biggest of these is a change to task-oriented processing. This change lets Plaso use much less memory during processing, and avoids a lot of potential pitfalls with concurrency and reliability. These changes should be pretty transparent to most users of Plaso.
As foreshadowed in the Freyja release, ZeroMQ is now the default queue implementation used by Plaso. Again, this change should be mostly invisible to most users, but sets us up for scaling Plaso beyond one machine.
The ElasticSearch output plugin has received some attention from @berggren, with improved reliability and support for raw field output. Gná will be a bit nicer for those using Kibana and ElasticSearch to investigate timelines.
One final under-the-hood changes is adding some initial support for Efilter. Efilter is a flexible query and filter engine which supports transforming queries between languages which opens up some interesting possibilities for making querying Plaso stores faster. Thanks to @the80srobot for help getting this integration to happen.

New features

  • New parsers and plugins
    • Thanks to new contributor @MacleodKen, Gná has parsers for Kik and iMessage databases from iOS devices (and MacOS, in the case of iMessage).
    • Plaso now also parses Twitter iOS databases, thanks to new contributor @aguilajesus.
    • @rgayon has added a parser for Docker log and config files, bringing Plaso into the exciting new world of containerization and “the cloud”.
    • Plaso Gná now parses events from Zsh history files, to help reconstruct user activity.
    • DC3 have added several Windows Registry plugins, creating more useful events for networks, network drives and the Winlogon key.
    • We also have DC3 to thank for the KML output plugin, to enable visualizing Plaso events that have a location component.
    • The RecycleBin parser has been updated to support different format used by Windows 10.
    • At long last, you can now add NSRL data to your Plaso timeline, to help filter out irrelevant events. The the nsrlsvr analysis plugin will tag events in a timeline by querying an nsrlsvr instance.
  • Yara
    • Plaso now supports matching file content with Yara rules. Point --yara-rules-path to a file of yara rules when you run log2timeline, and Plaso will set the attribute yara_match on events from any files that match the rules.
  • SQLite WAL parsing
    • Courtesy of DC3, Plaso’s SQLite parser (and thus, all the SQLite plugins) now support reading SQLite Write-Ahead-Log files, resulting in more events being retrieved from SQLite databases.
  • Syslog
    • The Syslog and SELinux parsers have been completely rewritten to improve speed, and expand coverage to different syslog formats.
    • The Syslog parser now supports plugins, enabling more meaningful processing of events from applications that log via syslog. At present, there’s plugins for logs generated by ssh and cron. If there are other apps that log to syslog that you’d like to see Plaso process, please create an issue, or send in some code!
  • Psort
    • Psort has been migrated to the same task-oriented processing model that log2timeline uses, which allows for more intuitive behaviour. It’s no longer necessary to run psort once to tag a storage file, and again to get the tags in the output. Running psort with analysis plugin will now include the results in any output.

What we broke this time

In previous versions of Plaso, we’ve advised that new releases might not be backwards-compatible with storage files generated with older version of log2timeline.py In a slight change, we can be quite categorical about Gná  - it does not support old storage files at all. Expect this to continue in future releases as well.


Some of the core changes we’ve made have necessitated some user-visible changes. One you’ll run into very early on is that psort now doesn’t output to standard output by default. Use the -w option to output to a file instead. So if you're used to:
psort.py storage.plaso > output.txt you'll have to do psort.py -w output.txt storage.plaso instead.


Due to the task processing refactor mentioned above, log2timeline will use a bit more disk space during processing than previously. If this causes you problems, try out the new --temporary_directory flag. Point it to faster storage (SSD, RAMDisk) to improve processing speed, or a volume with more capacity if you’re running out of space.

What we’re planning next

Gna brings a lot of changes, but we have even more in store for the future. In the next Plaso release, we plan to have some more core changes to the way Plaso internally stores event information. This change will enable a more substantial change to how Plaso stores all its results, to dramatically improve performance and expressiveness.
The other big change will be greatly improved support for forensic artifacts, again improving expressiveness, while also improving what Plaso can report about the data it’s analyzing.

Where to get Plaso 1.5?

See Plaso's Users' Guide and if you run into problems take a look at the Installation Problems page on the Plaso wiki, to see if other people have seen the issue before. If nothing there helps, ask for help on the discuss mailing list: log2timeline-discuss@googlegroups.com.


For those investigators Plaso on non-internet connected systems, we’ve added a guide for running Plaso from a Docker container, which is a handy way to bundle up all of Plaso’s requirements in a single place. Check out the instructions here: https://github.com/log2timeline/plaso/wiki/Installing-with-docker

One more thing

While we hear or read a lot of anecdotes about how people use Plaso (often when it's misbehaving), we’d really like to get a better idea of the features people use, to work out where we should focus our development efforts. To that end, we’ve made a survey to help us work this out. If you could fill it out, it’d be a great boon for the whole project.


Be mindful before raising issues and asking questions and read the troubleshooting documentation. Your question may be very well be answered there.

That’s all for now, happy investigating!

Comments

  1. UnknownSeptember 26, 2016 at 11:14 PM

    Great Job! Thanks you for your hard work.
    I have one question, is the elasticsearch output compatible with Kibana 4 and ES 2.x? And also, do you know if it will be compatible with ELK 5 (still not released, but got me wondering)

    ReplyDelete
    Replies
    1. Joachim MetzApril 11, 2017 at 11:31 PM

      I opt to ask this question log2timeline-discuss@

      Delete

Post a Comment

Popular posts from this blog

Parsing the $MFT NTFS metadata file

By
Background Numerous articles have been written about parsing the $MFT NTFS metadata file before. Some of these explain the technique itself [ 1 ] and others mostly focus on how to use tools. Rarely do articles mention the pros and cons of the technique from a digital forensics perspective [ 2 ].  What is $MFT parsing?  For readers not familiar with the NTFS file system, MFT stands for Master File Table. This table is stored in a file system metadata file, meaning that as far as the file system is concerned it is a file, however that the file contains file system metadata not “regular” content. The name of this file is “$MFT”.  The MFT consists of a sequence of (predetermined) fixed-size entries (or MFT entries). These entries are typically 1024 bytes in size, however the size is defined in the NTFS volume header and in a “regular” MFT entry itself.  The first 4-bytes of a “regular” MFT-entry (or record) starts with the signature “FILE”. The MFT entry can...
Read more

Incident Response in the Cloud

By
Many organizations have begun moving a majority of their services towards the cloud in recent years. As a result, attackers have shifted their focus towards the cloud. This has resulted in new techniques and methods specifically designed to compromise cloud infrastructure, like the recent SALTSTACK vulnerability that was widely exploited [ 1 ]. Therefore it is critical for these organizations to have an incident response team that understands the new risks attached to cloud and how cloud can make incident response easier or harder.  In this blog post we will walk you through each phase you may encounter in traditional incident response and highlight the differences when adopting cloud computing. It is aimed towards both those who are new to incident response and cloud computing. We’ve included insights that will benefit organizations taking that step towards cloud who want to ensure they are prepared to respond efficiently to cloud incidents.  Traditional Incident Response Be...
Read more

Container Forensics with Docker Explorer

By Anonymous
  Introduction As previous blog posts on cloud forensics have noted, applications are increasingly being deployed using container orchestration frameworks such as Kubernetes, especially in cloud environments.  Similar to traditional deployments on physical servers or virtual machines (VMs), when a containerised application has a security issue it can lead to a compromise of the underlying compute architecture. In the case of container deployments this means a compromise of the container itself, the container host, or even a wider cluster compromise via abuse of orchestration tools. Often digital forensics is required to establish what went wrong and remediate any issues. This article will provide an introduction to container forensics with Docker Explorer by working through a scenario involving a compromised container running within a Kubernetes cluster. Although Kubernetes is briefly mentioned, this article will focus on analysis of an individual container rather than the wi...
Read more