Simple Complete Smart Voice Control and Voice Feedback Systems

I successful build Voice Control and Voice Feedback Systems for my Smart Home Automation.

It might sound like the stuff of “Star Trek” or “Iron Man” fiction, but it is completely attainable.

Now, I can “discuss” with my smart home.

For example:

  1. Every morning, my smart home will automatically say “Good Morning” to me and brief me today’s news while I am having breakfast if it detect me sitting on “breakfast” chair.

  2. Just before going to sleep, I can ask my smart home to check status of the house and it will report status of the house to me by voice.

Cost to build Voice Control System: $50
Echo Dot $50

Cost to build Voice Feedback System: $65 or $200/$250

  1. Raspberry Pi route: $65
    Raspberry PI 3 model B $35
    Power Supply $5
    Case $5
    SD memory card $10
    USB speaker with 3.5 mm jack $10

  2. Sonos route: $200/$250
    Sonos Play 1 $200
    Sonos Play 3 $250

I have 2 Sonos Play 3, 1 Sonos Play Bar but I did not use Sonos for Voice Feedback System as I did not want to turn on Sonos 24 hours/day. I used Raspberry PI 3 model B + USB speaker with 3.5 mm jack for Voice Feedback System.

have Alexa, have Sonos, have multiple Pi’s. more details please…

There are 3 ways to setup Voice Control System:

  1. Echo + Vera Beta Firmware Program for Amazon Alexa
    http://support.getvera.com/customer/en/portal/articles/2648086-vera-smarter-home-control-▾-alexa-skills-kit-▾-beta

  2. Echo + Amazon cloud
    http://forum.micasaverde.com/index.php?topic=34527.0

  3. Echo + Raspberry Pi + HA bridge server
    GitHub - bwssytems/ha-bridge: Home automation bridge that emulates a Philips Hue light system and can control other systems such as a Vera, Harmony Hub, Nest, MiLight bulbs or any other system that has an http/https/tcp/udp interface. This is a compact impl to run on small format computers. This is impl started from this project https://github.com/armzilla/amazon-echo-ha-bridge.

There are 2 ways to setup Voice Feedback System:

  1. Sonos + Sonos plug in + Google TTS service

  2. Raspberry PI + USB speaker + TTS listener server + Google TTS service

Setting up Voice Feedback System using Sonos is very simple.
http://code.mios.com/trac/mios_sonos-wireless-music-systems

Making a Sonos device 666 say something:
luup.call_action(“urn:micasaverde-com:serviceId:Sonos1”, “Say”, {Text=“The sun shines outside”, Language=“en”}, 666)

Setting up Voice Feedback System using Raspberry + USB speaker is not simple as setting up Voice Feedback System using Sonos but I chose Raspberry + USB speaker because of 3 reasons:

  1. Raspberry PI + USB speaker is cheaper than Sonos. It can be turn on 24 hours/day.

  2. HA bridge server and TTS listener server can be installed on the same Raspberry PI.

  3. I plan to use Raspberry PI for wifi fencing.

Steps to setup Raspberry PI 3 model B for Voice Control and Voice Feedback Systems:

Preparing steps:

  1. Download latest raspbian into SD memory card and install raspbian.

  2. Install win32diskimager on laptop.
    Win32 Disk Imager download | SourceForge.net
    This tool is very useful to backup Raspberry PI SD memory card to laptop before we try to install and change a lot of settings in Raspberry PI. If any setting is wrong or Raspberry PI is hang, we can restore SD memory card from last working image, no need to re-install everything from beginning.

  3. Set static IP address for Raspberry PI. This step is very tricky. We need to backup SD memory card before we try this step. From this step onward, I will assume Raspberry PI static IP address is 192.168.1.5.

Setup Voice Control System

  1. Install HA bridge
    GitHub - bwssytems/ha-bridge: Home automation bridge that emulates a Philips Hue light system and can control other systems such as a Vera, Harmony Hub, Nest, MiLight bulbs or any other system that has an http/https/tcp/udp interface. This is a compact impl to run on small format computers. This is impl started from this project https://github.com/armzilla/amazon-echo-ha-bridge.

  2. Set HA bridge listen to 8080 port

  3. Open HA bridge admin page http://192.168.1.5:8080/ and build bridge devices for Vera devices and Vera scenes

  4. Ask Echo to discover devices: “Alexa, discover devices”. After this step, we have Voice Control System.

Setup Voice Feedback System

  1. Install mplayer
    open terminal
    pi@raspberrypi:~$ sudo apt-get install mplayer

  2. Connect USB speaker with 3.5 mm jack to Raspberry PI. Open youtube and make sure we can hear sounds from USB speaker. I spent 2 hours to make Raspberry PI use Echo Dot as a bluetooth speaker. However, bluetooth on linux is not reliable. My Raspberry PI could only connect to Echo Dot but could not output sound to it. After 2 hours struggling with linux-bluetooth headache, I gave up the “saving $10” idea and used cheap USB speaker with 3.5 mm jack.

  3. Setup Google TTS service
    open terminal, edit speech.sh file
    pi@raspberrypi:~$ nano speech.sh
    copy and paste the following to speech.sh. Ctrl X and Y to save the speech.sh

#!/bin/bash
say() { local IFS=+;/usr/bin/mplayer -ao alsa -really-quiet -noconsolecontrols “http://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&q=$*&tl=en”; }
say $*

  1. change execute mode for speech.sh
    pi@raspberrypi:~$ chmod u+x speech.sh

  2. Test Google TTS service
    pi@raspberrypi:~$ ./speech.sh Test text to speech

If we can hear “Test text to speech” output from USB speaker, we have successful completed setup Google TTS service.

Ahhhh. Now I get it. I’ve already have a similar setup with Vera using Sonos to tell me the weather when I switch on the bathroom light between 6 and 7Am on work days. Vera also uses Sonos to tell me if a door is opened (as one of my K9’s has figured out how to open the back door) and some other stuff. I can use Alexa to control lights and Sonos etc. Now even easier with the Amazon Wand (pay $20 + tax, get $20 credit)

I thought you had figured something out making it more interactive than currently available. I’m surprised Google TTS is working for you on Sonos. I could never get it reliable. The Microsoft TTS worked great untl MS shuttered it./ I’m using a local TTS now as although responsivevoice is good, the pauses it puts into longer speech started to annoy me.

[quote=“BOFH, post:6, topic:196507”]Ahhhh. Now I get it. I’ve already have a similar setup with Vera using Sonos to tell me the weather when I switch on the bathroom light between 6 and 7Am on work days. Vera also uses Sonos to tell me if a door is opened (as one of my K9’s has figured out how to open the back door) and some other stuff. I can use Alexa to control lights and Sonos etc. Now even easier with the Amazon Wand (pay $20 + tax, get $20 credit)

I thought you had figured something out making it more interactive than currently available. I’m surprised Google TTS is working for you on Sonos. I could never get it reliable. The Microsoft TTS worked great untl MS shuttered it./ I’m using a local TTS now as although responsivevoice is good, the pauses it puts into longer speech started to annoy me.[/quote]

Local TTS or Google TTS?

Local TTS does not need internet but it can only speak a few languages. I tried local pico TTS but did not like pico voice.

Google TTS needs internet but it can speak many languages. I tried to program Vera speak Vietnamese by changing speech.sh as below:

#!/bin/bash
say() { local IFS=+;/usr/bin/mplayer -ao alsa -really-quiet -noconsolecontrols “http://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&q=$*&tl=vi”; }
say $*

I have 2 tts functions: tts and tts_vi. My wife speaks Vietnamese, my sons speak English. For events that need my wife attention, I programed Vera to talk to my wife in Vietnamese by calling tts_vi function. For events that need my sons attention, I programed Vera to talk to my sons in English by calling tts function. I can not do that by using local TTS, I can only do that by using Google TTS.

What do you mean “more interactive than currently available”?

Echo Dot is just dumb hardware support for Voice Control System. Raspberry PI + USB Speaker is just dumb hardware support for Voice Feedback System.

Behind Echo Dot is Amazon’s Big Data Cloud Computers which provide software support for Echo Dot. Therefore Echo Dot can interact with more AI brain.

However, behind Raspberry PI + USB Speaker is dumb Vera, no Big Data Cloud Computer, no AI brain. We need to build more scenes, more triggers, more events, more lua program logic to make our Vera Smart Home more interactive.

Fortunately, we can program Raspberry PI + USB Speaker to talk to Echo Dot to make use of Amazon’s Big Data Cloud Computers (I put Raspberry PI + USB Speaker near to Echo Dot).

For example:

  1. This is dump “Good Morning” scene:
    tts(“Good Morning”)

  2. This is more interactive “Good Morning” scene:
    local GoodMorning = “Good morning " … os.date(”%A") – Good morning Monday
    tts(GoodMorning)
    tts(“Alexa, what is news today?”)

Another example:

Groceries are put on top of a pressure mat sensor. When pressure mat sensor detects groceries weight is reduced below threshold, it will trigger “Order Groceries” scene. Vera will use Raspberry PI + USB Speaker to talk to Echo Dot tts(“Alexa, order more groceries”)

  1. Backup SD memory card

  2. Install node server
    How to Install Node.js on Your Raspberry Pi - Jeremy's Raspberry Pi Blog

cd /usr/src
sudo wget http://nodejs.org/dist/v0.8.16/node-v0.8.16.tar.gz

sudo tar xvzf node-v0.8.16.tar.gz
cd node-v0.8.16

sudo ./configure
sudo make
sudo make install

sudo make test

nano test.js

// Load the http module to create an http server.
var http = require(‘http’);

// Configure our HTTP server to respond with Hello World to all requests.
var server = http.createServer(function (request, response) {
response.writeHead(200, { “Content-Type”: “text/plain” });
response.end(“Hello World\n”);
});

// Listen on port 8000, IP defaults to 127.0.0.1
server.listen(8000);

// Put a friendly message on the terminal
console.log(“Server running at http://127.0.0.1:8000/”);

And save it. Then run the follow

sudo node test.js

And then point your web browser to http://127.0.0.1:8000 and you will see a hello world! message.

  1. Setup custom node.js to listen to TTS call from Vera:
    http://forum.micasaverde.com/index.php?topic=14132.0

nano node.js

var net = require(‘net’);
var cp = require(‘child_process’);
/*

  • Callback method executed when a new TCP socket is opened.
    /
    function newSocket(socket) {
    socket.write(‘Welcome to the Telnet server!’);
    socket.on(‘data’, function(data) {
    receiveData(socket, data);
    })
    }
    /
  • Cleans the input of carriage return, newline
    */
    function cleanInput(data) {
    return data.toString().replace(/(\r\n|\n|\r)/gm,“”);
    }

/*

  • Method executed when data is received from a socket
    */
    function receiveData(socket, data)
    {
    var cleanData = cleanInput(data);
    var speakCmd = '/home/pi/speech.sh ';
    var speakData = “”;
    switch(cleanData)
    {

              case "@frontdoor":
                      speakData = 'There is movement detected at the front door.';
              break;
              case "@sidehouse":
                     speakData = 'There is movement detected at the side of the house.';
              break;
    
       
              default:
                      speakData = cleanData;
    
      }
      cp.exec(speakCmd + speakData);
    
      var currentTime = new Date()
    

    var hours = currentTime.getHours()
    var minutes = currentTime.getMinutes()
    console.log(hours + “:” + minutes + " " + cleanData);
    }

// Create a new server and provide a callback for when a connection occurs
var server = net.createServer(newSocket);

// Listen on port 5150
server.listen(5150);

  1. Test node + TTS
    cp.exec(‘/home/pi/speech.sh Welcome.’);

  2. Start node listener
    sudo node node.js

  3. Put function tts in Vera’s start up lua:

SS_SID = “urn:micasaverde-com:serviceId:SecuritySensor1” – Security Sensor Service ID
SP_SID = “urn:upnp-org:serviceId:SwitchPower1” – Switch Power Service ID

function turnOnLight(devID)
local lightStatus = luup.variable_get(SP_SID, “Status”, devID)
if (lightStatus == “0”) then
luup.call_action (SP_SID, “SetTarget”, {[“newTargetValue”] = 1}, devID)
end
end

function turnOffLight(devID)
local lightStatus = luup.variable_get(SP_SID, “Status”, devID)
if (lightStatus == “1”) then
luup.call_action (SP_SID, “SetTarget”, {[“newTargetValue”] = 0}, devID)
end
end

function tts(text)
local socket = require(“socket”)
host = “192.168.1.5”
local tcp = assert(socket.tcp())

if ( tcp ~= "nil" and tcp ~= nill) then
    tcp:settimeout(0.5)
    tcp:connect(host, 5150)
    --tcp:send("@frontdoor\r")
    tcp:send(text)
    tcp:close()
end

end

function log(message, clear)
local socket = require(“socket”)
local time = socket.gettime() or os.time()
local tms = string.format(“.%03d “,math.floor (1000 * (time % 1)))
local stamp = os.date(”%d %b %Y %T”,math.floor(time)) … tms
local mode = “a+”
if clear then mode = “w+” end
local file = io.open(“/www/log.txt”, mode)
file:write(stamp … (message or “”) … “\n”)
file:close()
end

– init global functions so that we can call these global functions in any scene.
tts(“Hello Vera”)
log(“Lua Startup”)

After this step, we have working Voice Control System and Voice Feedback System. We backup SD memory card to keep all our hardwork.

Our Vera now can give voice command to Echo Dot, we can focus on thinking how to program Vera to make use of Amazon’s Big Data Cloud Computers via Echo Dot.

We can also scale up the Voice Control System and Voice Feedback System by order more Echo Dot for each room, install more Raspberry PI + USB Speaker around our house.

For example: install a Raspberry PI + USB Speaker near main gate (in outdoor waterproof case). At night, if motion sensor detects any movement near the main gate, Vera can simply ask tts_maingate(“Who are you?”) then Vera can warn again if movement does not stop after minutes, tts_maingate(“This is private property. Any trespassing will be reported to Police”). We can not put expensive Sonos speaker outdoor, but we can put Raspberry PI + USB Speaker outdoor to enhance security system.

Looks cool! Will try it some day. Thanks for spending time on writing this howto. Regards Stefan

Skickat fr?n min SM-N910F via Tapatalk

Thank you for that code. I just obtained a Raspberry Pi Zero W and have some spare BT speakers. May have to give this a try…

Cost to build z-wave Pressure Mat Sensor: $60

PM1/PK - Stair Pressure Mat, 595x170mm $10 from element14.com. Bigger size mat cost more, $15, $20…

Fibaro door sensor $50 from amazon

Connect Pressure Mat to Fibaro door sensor, we will have z-wave Pressure Mat Sensor.

Some use case:

  1. Put Pressure Mat Sensor under door mat or stair mat, it can trigger event people standing in front of door or go up/down stair.

  2. Put Pressure Mat Sensor on sofa in living room, we can sit on it to watch TV without worry motion sensor turn off light because of no movement.

  3. Put Pressure Mat Sensor on a chair in dinner room, every morning our smart home can automatically say “Good Morning” to us and brief today’s news while we are having breakfast if it detect someone sitting on that chair.

  4. Put Pressure Mat Sensor on bed, Vera will know if someone is sleeping or not on that bed.

  5. Put a basket of groceries on Pressure Mat Sensor, Vera can give voice command to Alexa to order more groceries if groceries weight is reduced below threshold.

I’ve been looking at other pressure mats for similar applications. There is a nice thread on here about bed occupancy sensors using several different kinds of mats. I’ll look into these as well. Thanks.

@Iron Man - These tutorials are awesome! Thanks for taking the time to share…

Using Raspberry PI to build Wififencing System

Vera UI7 Geofencing is not reliable and it does not work with more than one mobile phone, so I decided to build Wififencing System to work with all our family mobile phones. Our family has 3 mobile phones: Daddy phone, Mommy phone and Son phone.

  1. Setup static IP address for all mobile phones
    For each mobile phone, open wifi setting on each mobile phone, note IP address of each phone.
    For example:
    Daddy phone IP Address 192.168.1.129
    Mommy phone IP Address 192.168.1.172
    Son phone IP Address 192.168.1.105

Login to main router admin page at http://192.168.1.1 and search for each phone IP address in the list of connected devices, note MAC address of each phone.
For example:
Daddy phone MAC Address 00:00:00:00:00:00
Mommy phone MAC Address 11:11:11:11:11:11
Son phone MAC Address 22:22:22:22:22:22

Goto router’s DHCP Server setting, manual assign IP address for each phone’s MAC address, i.e:
MAC address 00:00:00:00:00:00 assign to IP address 192.168.1.129
MAC address 11:11:11:11:11:11 assign to IP address 192.168.1.172
MAC address 22:22:22:22:22:22 assign to IP address 192.168.1.105

After this step, whenever our mobile phones join our wifi network, each phone will have a static IP address.

  1. Install nmap on Raspberry PI
    open terminal
    sudo apt-get install nmap

  2. Create wififencing.sh
    nano wififencing.sh

copy and paste the following into wififencing.sh, Ctrl X to save it.

#!/bin/bash

seconds between checks

WATCHDOG_SLEEP_SEC=15

Vera IP Address

Vera_IP=“192.168.1.2”

Daddy phone IP Address

IP_ADDRESS_1=“192.168.1.129”

Mommy phone IP Address

IP_ADDRESS_2=“192.168.1.172”

Son 1 phone IP Address

IP_ADDRESS_3=“192.168.1.105”

Son 2 phone IP Address

IP_ADDRESS_4=“0.0.0.0”

while sleep $WATCHDOG_SLEEP_SEC
do

Obtain currently set Mode from Vera

Current_Mode=$(curl -m 1 -s “http://$Vera_IP:3480/data_request?id=variableget&Variable=Mode”)

Only apply this script when Mode is NOT set to Night Or Vacation

if [ “$Current_Mode” = “1” ] || [ “$Current_Mode” = “2” ]
then
if (nmap -sn $IP_ADDRESS_1| grep “Host seems down”) && (nmap -sn $IP_ADDRESS_2| grep “Host seems down”) && (nmap -sn $IP_ADDRESS_3| grep “Host seems down”)
then
echo “No one is home. Set mode to Away. But first check if it is not already set.”
if [ “$Current_Mode” != “2” ]
then
echo “Mode was set to $Current_Mode. Changing to Away.”
curl -m 1 “http://$Vera_IP:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=SetHouseMode&Mode=2” > /dev/null
fi
else
# echo “Someone is home. Set mode to Home. But first check if it is not already set.”
if [ “$Current_Mode” != “1” ]
then
# echo “Mode was set to $Current_Mode. Changing to Home.”
curl -m 1 “http://$Vera_IP:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=SetHouseMode&Mode=1” > /dev/null
fi
fi
fi
done

  1. change mode for wififencing.sh
    chmod u+x wififencing.sh

  2. run wififencing.sh
    ./wififencing.sh

After this step, we have a wififencing system working 24/7 to monitor 3 mobile phones. Once it detects all 3 mobile phones disconnected from our wifi network, it will set Vera mode to Away mode. Once it detects any of 3 mobile phones joined our wifi network, it will set Vera mode to Home mode.

If your WiFi router runs DD-WRT, there is a script checking the presence of the MAC address(es) of the device(s) being present on the network. Determining home or away. It works on any Ui version. It’s posted in the forums here. I’ve used it for several years and I still use it as a backup for Vera’s GeoFencing…

I got the script from the forum. The script is for router with DD-WRT firmware, I modified it to make it works on Raspberry PI. Here is the script for router:

#!/bin/sh

seconds between checks

WATCHDOG_SLEEP_SEC=10

Vera IP Address

Vera_IP=“192.168.0.2”

MAC Address 1

MAC_ADDRESS_1=“00:00:00:00:00:00”

MAC Address 2

MAC_ADDRESS_2=“00:00:00:00:00:00”

MAC Address 3

MAC_ADDRESS_3=“00:00:00:00:00:00”

while sleep $WATCHDOG_SLEEP_SEC
do

Obtain currently set Mode from Vera

Current_Mode=$(curl -m 1 -s “http://$Vera_IP:3480/data_request?id=variableget&Variable=Mode”)

Only apply this script when Mode is NOT set to Night Or Vacation

if [ $Current_Mode == ‘1’ ] || [ $Current_Mode == ‘2’ ] ; then

if !(iwinfo wlan0 assoclist| grep -Fq $MAC_ADDRESS_1) && !(iwinfo wlan0 assoclist| grep -Fq $MAC_ADDRESS_2) && !(iwinfo wlan0 assoclist| grep -Fq $MAC_ADDRESS_3) && !(iwinfo wlan1 assoclist| grep -Fq $MAC_ADDRESS_1) && !(iwinfo wlan1 assoclist| grep -Fq $MAC_ADDRESS_2) && !(iwinfo wlan1 assoclist| grep -Fq $MAC_ADDRESS_3)
then
#echo No one is home. Set mode to Away. But first check if it’s not already set.
if [ $Current_Mode != “2” ] ; then
#echo Mode was set to $Current_Mode. Changing to Away.
curl -m 1 “http://$Vera_IP:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=SetHouseMode&Mode=2” >/dev/null 2>&1
fi
else
#echo Someone is home. Set mode to Home. But first check if it’s not already set.
if [ $Current_Mode != “1” ] ; then
#echo Mode was set to $Current_Mode. Changing to Home.
curl -m 1 “http://$Vera_IP:3480/data_request?id=lu_action&serviceId=urn:micasaverde-com:serviceId:HomeAutomationGateway1&action=SetHouseMode&Mode=1” >/dev/null 2>&1
fi
fi

fi

done

When I program Raspberry PI speak to Echo Dot, for example:
tts(“Alexa, what is news today?”)

Raspberry PI spoke too fast, Echo Dot could not catch up to understand the voice command. So I put 3 full stops after Alexa and Echo Dot could understand the voice command.
tts(“Alexa… what is news today?”)

Is there a plugin for Vera to enable this or only a script?

The script uses the Multiswitch plugin on Vera to toggle a button on that MultiSwitch for each network device being on or offline. Which can be used as a trigger for scenes et al.

My 10 year old son is giving voice control our smart home and getting voice feedback from our smart home.

[url=https://youtu.be/IzWp-bH9Bf4]Smart Home Vera + Echo Alexa + Raspberry Pi - YouTube