HOWTO: Event logging to database

Here’s a little guide to set up event logging using the standard Vera notification process.

Background:

The standard Vera notification process uses emails and/or text messages to notify you of anything that is going on, provided you have added notifications to the devices configured. When a notification needs to be sent, Vera calls a server at micasaverde with a number of parameters, and that server then takes care of the actual medium of sending. Although klunky, it does the job, but emails are “expensive” (lot of overhead for a simple notify) and text messages are unreliable. Not to mention, if you want to keep any kind of history, both methods are not really suitable for storing/parsing.

Vera offers a little known mechanism called ‘Alternate Event Server’. Using this method we are going to be able to receive notifications as they happen and in a format easily accessible to store them into a database.

Pre-requisites:

  • Vera.
  • Webserver with https listening. This can be apache2, lighttpd, whatever you fancy, but the important bit is that it supports url rewriting and php/mysql. The guide uses apache2.
  • MySQL database.

Setup:

I am not going to describe how to install/setup your own webserver as there are many, many guides on the internet covering this already. What is important that in order to continue with the next steps is that you have a webserver that listens on port 443 (https) and is able to serve a page from it.

First we need to create the database and tables which will contain our notification data. Start a connection as root to your database:

[font=courier]$ mysql -uroot -p
Enter password:
mysql>[/font]

Once presented with the prompt, enter the following commands to create the database:

[font=courier]mysql> create database vera;
mysql> create user ‘vera’@‘localhost’ identified by ‘vera’;
mysql> grant all privileges on vera.* to ‘vera’@‘localhost’;
mysql> flush privileges;
mysql> quit;
$[/font]

Download the attached zip file and extract it in the document root for the secure content server. You’ll find 4 files inside:

[font=courier]$ ls -la
-rw-r–r-- 1 quinten quinten 51 Jun 12 13:01 .htaccess
-rw-r–r-- 1 quinten quinten 2089 Jun 12 13:01 alert.php
-rw-r–r-- 1 quinten quinten 352 Jun 12 13:01 alert.sql
-rw-r–r-- 1 quinten quinten 160 Jun 12 13:02 config.php
$[/font]

The [font=courier].htaccess[/font] file redirects the request Vera makes to the [font=courier]alert.php[/font] script. The [font=courier]alert.sql[/font] script contains the database definition and the [font=courier]config.php[/font] contains the configuration variables.

Edit [font=courier]config.php[/font] and set the variables correctly. Pay particular attention to the [font=courier]$vera[/font] variable which needs to point to your Vera’s IP address or name.

Now create the tables:

[font=courier]$ mysql -uvera -pvera vera < alert.sql
$[/font]

Further configuration:

If you’re like me, and do not want to receive an email everytime a notification is sent, you can circumvent this by creating a new user and NOT validate the email address that you’ve given it. Notifications will still work, but no emails will be sent out. Just ignore and delete the validation email that is sent to the notification user.

Just discovered that simply defining a notification without assigning a user will still sent the event to the alternative event server… No need for any users to be created!

Now all we need to do is tell Vera to use our alternate event server by entering the following url inside a browser:

[font=courier]http://<your.vera.ip.address>:3480/data_request?id=variableset&Variable=AltEventServer&Value=<your.webserver.ip.address>[/font]

You will need to reload your Vera (hit the Reload button in the Vera web interface) for it to become effective.

Now for the events themselves.

Visit each device that you want to log notifications for and set up the notifications that you are after. The Name you give will find its way into the database as the description in the [font=courier]events[/font] table. The [font=courier]alert.php[/font] script will automatically create entries in the [font=courier]devices[/font] table using the device name that Vera holds.

[font=courier]mysql> select * from devices;
±—±---------------------+
| id | name |
±—±---------------------+
| 21 | Garage Door |
±—±---------------------+
1 rows in set (0.00 sec)

mysql> select * from events;
±—±-------±-------------------------------------±------±--------------------+
| id | device | description | value | tstamp |
±—±-------±-------------------------------------±------±--------------------+
| 1 | 21 | Garage door opened | 1 | 2013-06-11 15:50:04 |
±—±-------±-------------------------------------±------±--------------------+
1 rows in set (0.00 sec)[/font]

I hope this is helpful in any way, and I’m looking forward to any comments and improvements you may have.

Q.

Great write up. I was looking at this api the other day. Your guide will come in handy.

  • Garrett

I believe it will also record every Scene Trigger unless you are filtering these out.

thanks for sharing!

Notifications are on the device, so if a scene changes the state of a device, then yes, you will also get the notification for those.

The purpose of the above is not to provide alerts, but merely retain a history of events. The datamine plugin provides possibly the same information, but this keeps all my stuff neatly in the same database as I am using for all my other automation projects.

Scene triggers are fired when devices change their state.
This causes the Scene to run.

Notifications on a device are implemented as a mini scene in Vera with the specified notification description defining the trigger.

My point was that all Triggers, including Notifications will get captured.

Hi

Thanks to a reminder from @S-F i thought I’d check this out as i’d like another way to log events (i love dataMine but it keeps messing with my memory keys :wink:

UPDATE - I’ve just managed to solve the problem I originally posted here 5 mins ago so I removed it… - UPDATE

@Quinten a few questions for you…

You talk about telling Vera to use alternative event server? Does that stop any logging that was previously done, I’m just curious if this changes something crucial or if this is just an additional logging facility

How are you using this information, you mentioned you are using if for your other HA projects. I would be interested to see if you have a way to present this via a web page ?

Speaking of DataMine how can I set it to on Vera to capture just a change in a variable e.g last update?

With the event table created and some events set up, I kicked it off over night and have received notification emails, but nothing was populated into the tables.

mysql> show tables;
±----------------+
| Tables_in_vera |
±----------------+
| devices |
| events |
±----------------+
2 rows in set (0.00 sec)

mysql> SELECT * FROM devices;
Empty set (0.00 sec)

mysql> SELECT * FROM events;
Empty set (0.00 sec)

Any ideas ?

@parkerc

Any progress? Please keep the peanut gallery informed on your quest!

Hi @S-F

Sadly no progress, no entries seem to be going into the events table, nor is the devices table populated. I’m sure it is something obvious, but I can’t see it… Do you have it working?

No. I haven’t even tried. I’ve got too much on the table right now. My WHS runs IIS so I can just put it on there. I’m going to keep an eye on this thread and try to tackle it this winter.

So you can see all the mistakes I make :wink:

In theory this is a cool idea, however even looking at the wiki - http://wiki.micasaverde.com/index.php/AlternateEventServer - I’m possibly in over my head. I had an old NAS that runs MySQL and apache, so I thought what the heck, I’ll give it a go.

Not sure what it is but this is in the logs.

01 08/22/13 18:49:49.121 FileUtils::ReadURL 28/resp:0 size 0 https://192.168.1.111/alert?PK_AccessPoint=35102036&HW_Key=<longstringwhichihaveremovedincaseitisimportant&DeviceID=29&LocalDate=2013-08-21%2003:12:17&LocalTimestamp=1377051137&AlertType=3&SourceType=3&Argument=12&Format=&Code=CurrentTemperature&Value=18&Description=Conservatory%20Temp%20Drops%20Below%2020&Users=157707 <0x2c215680> 01 08/22/13 18:49:49.122 RAServerSync::SendAlert retries 27 failed https://192.168.1.111/alert?PK_AccessPoint=35102036&HW_Key=longstringwhichihaveremovedincaseitisimportan&DeviceID=29&LocalDate=2013-08-21%2003:12:17&LocalTimestamp=1377051137&AlertType=3&SourceType=3&Argument=12&Format=&Code=CurrentTemperature&Value=18&Description=Conservatory%20Temp%20Drops%20Below%2020&Users=157707 age: 142652 file: /etc/cmh/persist/a_1377046826_1377051137_1bff198

I’m about stumped if this is an access rights issue, also checking out the alert.php page it gives me this error.

Fatal error: Call to undefined function header_remove() in /var/www/ibox/alert.php on line 3

Line 3 is.

header_remove();

UPDATE - it seems I only have php version 5.2.9, and the entry header_remove(); may only work on 5.3.x versions and above.
UPDATE1 - By editing the php.ini I can remove the header being sent, so I think i can now remove the line header_remove();
UPDATE2 - With that line removed, a new error arrives

Fatal error: Class ‘mysqli’ not found in /var/www/ibox/alert.php on line 20

UPDATE3 - Yep, it seems my version of php does not have mysqli installed ! (You could not write this stuff) I could update the files so they use PDO but sadly I don’t know how to do that…

Sorry for the (very) late reply guys. Currently on my hols and have very limited access (had to do 10 days without internet, the cruelty of it!), but will pick this up when I return in a couple of days… hang on to something tight! :wink:

Right, am back in full swing… Can you give me an update on where you are with this now? Or have you abandoned it due to the lack of (uptodate) mysql/php bindings?

Hi @Quinten

I’d prefer to say it’s been placed on hold :wink:

Things are still as they were, the version of php I have does not seem to have mysqli , but does seem to have PDO (whatever that is). Upgrading the NAS linux version of PhP does not look straight forward to do. ?

I really like the idea of having everything captured in a relational table, (no disrespect to DataMine, as that’s great). Do you know if it’s possible for every change in a selected variable to be logged/captured

What NAS is it? It may just be as simple as updating a package, but it could be impossible if it is a closed system. What I would like to say is that you can test the workings of your web server by just copy and pasting the url out of your log file into a browser window, provided you have access to the webserver from there (i.e., you can see normal pages on it)

https://<your.webservers.ip.address>/alert?PK_AccessPoint=35102036&HW_Key=BlahBlah&DeviceID=29&LocalDate=2013-08-21%2003:12:17&LocalTimestamp=1377051137&AlertType=3&SourceType=3&Argument=12&Format=&Code=CurrentTemperature&Value=18&Description=Conservatory%20Temp%20Drops%20Below%2020&Users=157707

From that, the device id and description are used (along with the timestamp). Once you then have entries in your database, you can set the alternative event server as per the instructions…

As to your last question, anything that can be set up as a notification, can be stored in the database. The alternative event server is informed at the same time as the event server at MicasaVerde, so it does not replace it, but mirrors it.

Hi Quinten

It’s an old NAS, the brand name is eTRAYZ and it looks to be running php 5.2.9 which does not seem to support mysqli, and that’s why I think it is not updating the tables.

I looked on the php site and it did not look straight forward thing to update a package ?

Can you copy&paste the following into a file on the NAS and access it in a web browser? What does it do?

<?php

// Make a MySQL Connection
mysql_connect("localhost", "YOURUSERNAME", "YOURPASSWORD") or die(mysql_error());
mysql_select_db("YOURDATABASENAME") or die(mysql_error());

// Retrieve all the data from the "events" table
$result = mysql_query("SELECT * FROM events")
or die(mysql_error());  

// store the record of the "events" table into $row
$row = mysql_fetch_array( $result );

// Print out the contents of the entry 
echo "Device: ".$row['device'];
echo " Description: ".$row['description'];

?>