Is the code for the MiOS z-wave module available somewhere, or is that proprietary?
Not only is Z-Wave proprietary, including MiOS’s implementation of it, vendors are under NDA.
futzle,
Thanks again for your quick response!
I’m assuming that when a z-wave device says that its parent is Z-Wave, that means the code that is operating the device is in the MiOS Z-Wave module. Is that correct?
Does anyone know where the code that operates all of the z-wave devices is kept?
That too is proprietary, inside of the LuaUPnP executable, for which there is no source.
Z-Wave, as a protocol, is pretty locked-down. A few people have reverse-engineered bits of it (search the Web and you’ll find it), but if you are looking to have this kind of low-level access to Z-Wave devices in Vera then you have a lot of original research to do.
So, if I understand what you’re telling me, if I want to have any control over the way z-wave devices are operated, the data is presented to the user, or the user interfaces with the devices, I would have to switch to an open-source platform like OpenZWave? Is that correct, or am I missing something?
Maybe the original questions needs clarification… What is the intended or desired outcome that you need the details for?
Luup commands
http://wiki.micasaverde.com/index.php/Luup_Requests
As a simple example, I just purchased 3 Aeon 4-in-1 sensors to control various devices around my house. I installed the devices and ended up with 12 new tiles on my dashboard, each containing a single bit of information. I have modified the standard MotionSensor1 device descriptor to show all four pieces of information from a single sensor on a single tile. I also don’t want the Aeon sensor handling any kind of security functions, because I have a separate set of plugins that handles all security functions uniformly for all types of devices. So I removed the “Armed” and “Bypass” functions from the MotionSensor1 descriptor.
While this solution is an acceptable tradeoff for me, it means that all devices that use the standard MotionSensor1 descriptor are going to have my new format. Not exactly what I intended.
It does not seem that any of these kinds of changes should be proprietary to z-wave or covered by the NDA. It would seem that z-wave devices could use the same kind of implementation files as any other kind of device. Maybe MCV just hasn’t separated out the proprietary parts of the z-wave network from the standard device description parts, or maybe they have made a business decision that they want to control all aspects of z-wave device operation. In any case, this seems to make z-wave devices much less usable than any other kind of device.
trakker, you’re sort of right, and sort of looking at it wrong (which is why you’re asking what I’d call the “wrong” questions, and why you feel you’re getting the “wrong” answers from us).
Vera is two things: it’s an HTTP-to-Z-wave gateway, receiving requests over the port 3480 HTTP API (see the Luup Requests page that Brientim pointed to). This layer has no UI, it just presents an abstraction of Z-Wave that borrows a lot from UPnP. This layer is implemented in C, it’s MCV’s private code, and you can’t really mess with it. You shouldn’t need to mess with it, either.
The other thing that Vera is is a web browser interface, presenting the aforementioned abstraction on the dashboard that you see when you point your browser at your Vera’s IP address. Since this layer is mostly JavaScript, XML and JSON files, you can easily see this code and change it (though it is still owned by MCV and subject to copyright). This is the layer that you are messing with by editing the JSON files.
If you don’t like the UI (you’re not the only one), you can make small tweaks to the JavaScript and JSON, but you do risk breaking the UI JavaScript code if you start editing files, as you’ve rightly pointed out. In essence you are seeking the same kind of customizability as the people who want to have different icons for individual switches in the dashboard (see the many previous discussions on the forum). This level of customizability is not available in the Vera UI. But you can make your own UI that interfaces straight with the port 3480 API. That’s basically what the mobile device controllers (HomeWave, SQRemote, AutHomation, iVera and others) do. These are all implemented the same way (see the UI_Simple wiki page that Brientim pointed to). You could do the same, if you don’t like any of the ones that are already available.
Notice how you’re a couple of levels of indirection above the Z-Wave protocol that you started asking about. So while my answers were correct, they’re probably not useful to you.
futzle,
Thanks again for all of the information. I’m trying hard to keep up. Really, I am. ;D It sounds like you’re telling me that the reason z-wave devices have no implementation files is a function of the UI, and has nothing to do with the z-wave protocol or the NDA. You also said that all of the UI code is readily viewable and changable. Where is the code that is creating the four tiles for each of my Aeon sensors and that is responding to user input?
Going back to my example above, it sounds like you’re telling me that if I wanted to create a plugin for my Aeon sensor that is just like the ones I have created for all of my non-zwave devices, I would have to rewrite the entire UI. Am I understanding you correctly, or am I lost again?
It sounds like you're telling me that the reason z-wave devices have no implementation files is a function of the UIHas nothing to do with UI. There is no implementation file because Z-Wave devices are managed by their parent devices, which is called ZWave, and this device is internal to Vera because of the NDA @futzle has mentioned.
Each Z-Wave device supported by Vera has a device type. Each device type has a corresponding D_XXXX.json file. This describes the Events, and user interface for ALL devices that share the device type. Vera does NOT support customization at the instance level (for a specific device).
No, that’s another wrong question (sorry). Here’s the right questions to ask:
Where is the code that is creating four devices for each of my Aeon sensors? Where is the code that is presenting each of the four devices as tiles on the dashboard? Where is the code that responds to the clicks I make on the dashboard and translating that into Z-Wave commands?
To which the answers are:
The code that exposes Aeon all-in-one sensors as four devices is in LuaUPnP, it’s written by MCV and you can’t alter it. The code that identifies the Aeon Z-Wave device is in LuaUPnP, and the choice to make it four devices and not one is a choice made by MCV and not one that you can override. MCV’s attitude is that it’s better to present each aspect of the all-in-one as if it were a standalone device, for logical consistency with truly single-use Z-Wave devices. You may not like that, but it’s outside your power to change.
The code that presents the four devices as four tiles on the dashboard is in the D_.json files and associated JavaScript. You can hack these to your heart’s content but you’re only affecting the web interface (globally), you are having no effect on the Z-Wave device. If you remove a UI element from the D_.json file you aren’t stopping that functionalty from existing, you are only hiding it from the user. The user can still call the port-3480 interface directly, say, with a mobile phone app that doesn’t ever use the D_*.json files.
The code that reacts to the clicks that you make on the devices in the dashboard is in the D_*.json files and associated JavaScript. You can hack these a little bit, only as far as they invoke UPnP actions that go to the HTTP-on-port-3480 interface. From there on it’s inside LuaUPnP again and you can’t get to them.
About plugins: Plugins are Lua fragments that run inside of LuaUPnP, and can call back into LuaUPnP through a very limited interface (http://wiki.micasaverde.com/index.php/Luup_Lua_extensions). That interface doesn’t go anywhere near LuaUPnP’s Z-Wave code, which is why it’s not possible to make a plugin that controls a Z-Wave device. Or, to put it another way, LuaUPnP handles Z-Wave devices internally in C, so there’s no need (or place) for a matching implementation file. Here is where I agree with you that non-Z-Wave devices have more plugin-ability than Z-Wave devices do.
I couldn’t say whether the choice to do Z-Wave entirely inside of LuaUPnP is a feature of the Z-Wave NDA or a feature of MCV’s architectural choices with MiOS, but the effect is the same: when you add a Z-Wave appliance to Vera, you get what you get. Perhaps one device, perhaps many.
You do have control over whether the UI displays those devices to the user, but only if you are prepared to ditch the Vera dashboard interface and substitute your own. Personally I use the phone interfaces of HomeWave and AutHomation HD, where I can pick and choose which devices I want to include on the screen. On a web browser, you don’t have a lot of (any) options; no one has written a comprehensive web-browser skin other than the default dashboard one, although it’s technically possible.
Guys,
Thanks very much for all your help!!! I think I’ve got it. I don’t like it, but I’ve got it. 
I think I see OpenZWave in my future. 
I have played with Open Z-Wave. I wrote my own C# version as well. Many of the Z-Wave dongles, and I have tried a number, have their own quirks you have to deal with. You will not be able to control any Locks with Open Z-Wave.
That code has not been reverse engineered. And to access the Z-Wave source code you need to sign a NDA and pay $25K. So if you just want to control lights and react to sensors … you are good to go.
There are configuration details you will have to sort out for each device you wish to control.
(MCV has already done this and they map these to the UPnP device types)
The lack of Lock Support caused me to go with Vera. Also I did not really want to program at this low level of abstraction.
Richard,
Thanks for the heads up!
The alternative is to create your own device to capture the variable in the same way Variable Container is used, and hide the other devices. It may not achieve the full desired outcome but basically create a virtual motion sensor gives the flexibility is the as delivered design aspects.
I look at Open Z-wave awhile ago but there were some limitations and still heavily reliant on third party devices and reverse engineering or just http calls to Vera. The 3rd party iOS and Andriod apps delivered the same capability but not the same level of graphical presentation without the effort.
I sometime question the intent of this activity where the end state is the same functional capability as the start point. I appreciate consolidation and aggregation of data but for some simple on and off… That is not your requirement; however, it does raise itself every now and then and there sits Open Z-wave to fill the desire.
By combining all of your information and suggestions, I believe that I have solved my problem. I don’t know how general the solution is, but it seems to have addressed all of my concerns.
There is a file in MiOS called /etc/cmh/zwave_products_sys.xml that contains the definitions for all of the “custom” z-wave devices that MiOS knows about. The 8 lines in that file that pertain to my Aeon 4-in-1 sensor are as follows:
This file is refreshed each time the firmware is upgraded, so that any changes made to it are lost. However, there is another file that you can create called /etc/cmh/zwave_products_user.xml that appends to the first file, but is permanent (not refreshed). Unfortunately, the user file doesn’t override the sys file (as it should) so, in order to make changes, you must delete the relevant lines from the sys file. I deleted the above lines from the sys file and replaced them with a user file containing the following 9 lines:
This change accomplished several things. First, it allowed me to substitute a custom device descriptor file (D_Aeon4in1Sensor1.xml) in place of the standard one (D_MotionSensor1.xml). Second, it allowed me to turn off polling of the device (PollSettings=0). Third, it allowed me to automatically set up all of the configuration parameters in a way that is more appropriate for a USB powered device. Anyone that is planning on using the Aeon 4-in-1 Sensor for anything more than casual monitoring should be sure that the device is USB powered. Only then will it report all of its readings on a regular user-specified schedule, independent of whatever else may be happening on the z-wave network. I use the differences in temperature and humidity between physically-separated sensors to control a high-powered fan motor in the attic. Making sure that the sensor readings are reliable and synchronized is important to me.
The second phase of the process was to write a normal MiOS plugin for the Aeon 4-in-1 Sensor, just as I would have for any non-zwave device. Surprisingly (to me), the Implementation file ran and called the Startup routine, as usual. The only problem (as with all child devices), is that the Startup function is called with the parent’s device number. Since all z-wave devices are children of the ZWaveNetwork (device #1), the Startup routine is called with a “1”. Therefore, you have to come up with some clever way of determining which child device you’re trying to start up as. Once you’re over that hurdle, everything else works normally.
I used the Startup routine to activate a luup.variable_watch on all four of the sensor’s main variables. When any of them change, I recast the information into a format that (I believe) is more user-friendly. It is this reformatted information that is shown on the UI as specified in the D_Aeon4in1Sensor1.json file. I debated whether or not to reformat the information back into the same main variables that it came from (without creating any loops), but in the end I decided to use a separate set of variables. Any remote control software will still see the same services and variables that it would have seen in the original file, so I will leave the reformatting to them. Finally, I automatically made all of the children of this device invisible, so that they are (effectively) removed from all displays. I tried to actually delete them automatically in several ways, but was never successful.
In the end, I was able to create a single device that is automatically installed and automatically configured that displays all of its variables in just the way that I want them displayed. I’m sure that there are many good reasons not to do what I have done, but in the end I was able to accomplished all of my objectives with a minimum of fuss.
Nice going, trakker, and thanks for reporting it back to the forum. Let us know how it works for you in the long term.