How to perform real-time performance monitoring

If you have ever wondered how to perform real-time performance monitoring on a number of virtual hosts scattered across multiple locations, then install and use netdata application.

Overview

netdata is a low-overhead monitoring solution which focuses on current performance and displays gathered data using interactive web-based dashboard.

It is an ideal solution to keep an eye on real-time statistics using web-browser, but keep in mind that it is not designed to store historical data as it stores collected data in memory, by default it is an one hour of data points, stored and loaded again on restart.

It is a modern, beautiful, simple and very descriptive application.

Installation process

Install required dependencies.

$ sudo apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config curl

Download the source code.

$ git clone https://github.com/firehol/netdata.git --depth=1

Change working directory.

$ cd netdata

Build and install application using /srv/netdata prefix.

$ sudo ./netdata-installer.sh --install /srv/netdata

  ^
  |.-.   .-.   .-.   .-.   .  netdata
  |   '-''-''-''-'   real-time performance monitoring, done right!
  +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->


  You are about to build and install netdata to your system.

  It will be installed at these locations:

   - the daemon    at /srv/netdata/usr/sbin/netdata
   - config files  at /srv/netdata/etc/netdata
   - web files     at /srv/netdata/usr/share/netdata
   - plugins       at /srv/netdata/usr/libexec/netdata
   - cache files   at /srv/netdata/var/cache/netdata
   - db files      at /srv/netdata/var/lib/netdata
   - log files     at /srv/netdata/var/log/netdata
   - pid file      at /srv/netdata/var/run

  This installer allows you to change the installation path.
  Press Control-C and run the same command with --help for help.


[...]

-------------------------------------------------------------------------------

OK. NetData is installed and it is running.

-------------------------------------------------------------------------------

By default netdata listens on all IPs on port 19299,
so you can access it with:

http://this.machine.ip:19999/

To stop netdata, just kill it, with:

  killall netdata

To start it, just run it:

  /srv/netdata/usr/sbin/netdata


Uninstall script generated: ./netdata-uninstaller.sh
Update script generated   : ./netdata-updater.sh

  ^
  |.-.   .-.   .-.   .-.   .-.   .  netdata                          .-.   .-
  |   '-''-''-''-''-'   is installed and running now!  -''-'
  +----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+-----+--->

  enjoy real-time performance and health monitoring...

Configure application to listen only on localhost instead on every interface.

$ sudo sed -i -e "s/# bind to = \*/bind to = 127.0.0.1/" /srv/netdata/etc/netdata/netdata.conf

Configure application to store two hours of collected data instead of just one.

$ sudo sed -i -e "s/# history = .*/history = 172800/" /srv/netdata/etc/netdata/netdata.conf

Copy generated systemd service file.

$ sudo cp system/netdata.service /etc/systemd/system/

Reload systemd, enable service at boot and start it right away.

$ sudo systemctl daemon-reload

$ sudo systemctl enable netdata

$ sudo service netdata start

Install nginx http proxy server.

$ sudo apt-get install nginx

Create directory to store ssl certificate.

$ sudo mkdir /etc/nginx/ssl

Generate ssl certificate for an IP address.

$ sudo openssl req -subj "/commonName=$(ip address show dev eth0 scope global | awk '/inet / {split($2,var,"/"); print var[1]}')/" -x509 -nodes -days 730 -newkey rsa:2048 -keyout /etc/nginx/ssl/nginx.key -out /etc/nginx/ssl/nginx.crt

Generate credentials for basic access authentication (basic-user username, basic-pass password).

echo "basic-user:$(openssl passwd -crypt basic-pass)" | tee /etc/nginx/htpasswd

Generate minimal nginx virtual host configuration.

$ sudo cat <<EOF | tee /etc/nginx/sites-enabled/default
server {
  listen 8080;
  server_name default;

  auth_basic "Restricted access";
  auth_basic_user_file /etc/nginx/htpasswd;

  location / {
     proxy_pass http://127.0.0.1:19999/;
  }
}
EOF

Reload nginx configuration.

$ sudo systemctl reload nginx

Update process

Update is very simple, it depends on a shell script created during earlier installation process.

$ ./netdata-updater.sh

This shell script can be missing if you were an early adopter, so you need to pull changes and re-install application manually. Update shell script will be created automatically.

$ git pull

$ ./netdata-installer.sh --install /srv/netdata

Enjoy!