For a recent exchange we've been building, we've been using the ELK stack (Elasticsearch, LogStash, Kibana) for real-time event processing alongside Datadog for alert / process monitoring. It's worked swimmingly well.
So what's LogStash's role in all this, you ask? It makes our log files useful, which sounds boring but is truly awesome - especially for the trading community. Because when your log files are filled with FIX Protocol messages, you're going to have nightmares of SOH characters and deciphering garbage into meaningful data - just ask exchange operators and regulators. Also, LogStash is configurable to have multiple outputs so we can monitor it in Datadog, or query via Elasticsearch, or store it anywhere else we want. Effectively, this allows us to easily create a timeline of a transaction from its inception as a NewOrderEntry to a matched trade. As orders are placed, and a book builds, our filter sits there, deciphering each message as it's logged and redirecting it to a datastore of our choice. What used to cost millions of dollars of infrastructure, can now be implemented in a couple of days. Frankly, I'm surprised elasticsearch doesn't have a team of sales reps actively selling this to Financial Institutions.
Connamara has released a press release for the first version of the plugin, so I won't go further in explaining the utility of the filter here - check it out yourself. Instead, I'll supply a link to an internal presentation I put together to demonstrate the filter and how LogStash / the ELK stack works. Enjoy!
BTW - It's insanely easy to get started building a LogStash Filter. They even offer a tutorial.