northben's blog

As I've worked on many large-scale Splunk environments, a common problem I've noticed is that Splunk forwarders phone home to the deployment server (DS) too frequently. When a forwarder phones home to the DS more often than necessary, it wastes resources on the DS, and can prevent the DS from deploying apps to forwarders correctly.

If you are Splunk admin, consultant, or power user, you may find yourself referring to the docs frequently. I bookmarked the Splunk docs that I frequently use, and I am sharing these bookmarks for everyone to use.

To add these bookmarks to Chrome, open the bookmark manager and then click import bookmarks.

To add these bookmarks to Firefox, open the bookmark manager and then click import bookmarks from HTML.

As a Splunk administrator, have you ever needed to list out your saved searches in some way? Perhaps you need to know which searches might be accelerated, scheduled, or even real-time scheduled? Here's a quick dashboard to show this information.

The dashboard is available in this GitHub repo.

If you work with the Markdown markup language, it's only a matter of time before you need to escape the markdown formatting characters. For instance, pasting source code into Trello cards requires the text to be escaped or else certain characters of the source code will be interpreted as markdown.

When you're working with the stats command, it's often nice to rename the fields to drop the aggregatation type:

| makeresults count=3 
| streamstats count 
| eval foo="bar" 
| stats latest(foo) as foo latest(count) as count

This gets annoying because of all the extra typing involved, and it violates the DRY principal.

Instead, try this next time:

When you create an extraction in props.conf (a search-time field), you specify the name of the new field through a named capture group in the regular expression. For example:

EXTRACT-foo =  (?P<foo>\w+)

This configuration will create a field "foo" with a value of the first word found in the _raw field. You can also specify a field besides _raw in your extraction:

EXTRACT-foo =  (?P<foo>\w+) in host

This is a simple request, but the official docs are out of date.

1. Create a file: $SPLUNK_HOME/etc/apps/search/local/ui-prefs.conf
2. Create a search stanza like this:
   
   [search]
   dispatch.earliest_time = -7d@d
   dispatch.latest_time = now

Restart Splunk and you're good to go!

If you are having trouble getting a Linux (Ubuntu, in my case) guest OS to get a network address using "bridged networking", make sure you have installed "virtualbox-guest-additions-iso".

You can install this package through apt-get:

apt-get install virtualbox-guest-additions-iso

As soon as I installed this package, my Ubuntu 15.10 Server guest received a DHCP address and was connected to my network!

Indulge me for just a moment, and consider a future in which you choose a bank just as easily as you choose a gas station. Where banks charge competitive fees for the useful commodity services that they actually provide. Where the banks don't hold a monopoly on access to the financial system. Where individuals like you and I can autonomously contribute to the development and improvement of the financial system.

I just discovered that Simple XML dashboards in Splunk can be customized with JavaScript! That's right, the point and click dashboards can be customized in a very powerful way. Here's the official docs, and an official example.