HomeGenie Forum

Automation Program Plugins and Wizard Scripting => APP Contributions => Topic started by: mvdarend on October 18, 2016, 07:31:45 PM

Title: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on October 18, 2016, 07:31:45 PM
With help from raptorjr in creating the Ethernet part, I've put it together in a combined Serial/Ethernet gateway for MySensors v2.0

Supported Devices:
S_DOOR *
S_MOTION
S_LIGHT / S_BINARY
S_TEMP
S_HUM
S_UV
S_POWER *
S_DISTANCE *
S_LIGHT_LEVEL
S_LOCK *
S_IR *
S_WATER *
S_RGB_LIGHT *
S_GAS *

I_BATTERY_LEVEL *
I_ID_REQUEST
I_INCLUSION_MODE *

* Not tested

This list will be added to as I create support for new devices, if you want a certain device supported please let me know in this thread.

Settings

Device Type (Serial, Ethernet): Default Serial

Ethernet IP address: default 127.0.0.1

Ethernet Port: default 5003

Serial COM Port: default /dev/ttyUSB0

Serial Baud rate: default 115200

Inclusion Mode always on (True/False): Set this to false if you dan't want to add newly found devices to HomeGenie. (default True) Inclusion mode can also be activated from the Gateway (not tested)

Group Subdevices with ID < 10 (True/False): If you want to group multiple sensors within one HomeGenie widget (for example temp/humidity) set this to true and give the Child Sensors an Id below 10. All Child Sensors with an Id of 10 or higher will automatically get a separate HG widget (default True)

If you have an Actuator on a MySensor Node (eg. Switch, Light etc.) it must have it's own widget or HomeGenie cannot send commands to it, there are three options:

- Have the Actuator on its own MySensor Node
- Set 'Group Subdevices with ID < 10' to False
- Set 'Group Subdevices with ID < 10' to True and give the Actuator a ChildId of 10 or higher.

Any Improvements/Remarks greatly appreciated.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on October 18, 2016, 07:35:14 PM
I've also created a few custom Sensor types for HomeGenie, to get these to display properly you'll have to edit the homegenie.generic.sensor widget (until hopefully the next version of HG, a pull request has been submitted) you can do this as follows:

- Configure -> Widgets
- open the homegenie.generic.sensor widget
- Click on the Javascript tab
- Scroll down to line 154 and change it to the following:
From this:
Code: [Select]
if (!ctx.isUnknown) {to this:
Code: [Select]
if (!ctx.isUnknown || parameter.Name.indexOf("Sensor.") >= 0 ) {- Actions -> Save

You can test it by clicking the Run/Preview button and binding to an existing module.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: raptorjr on October 18, 2016, 07:59:17 PM
I really didn't do much, but thank you for mentioning me =)

But I will be using and testing it a lot, so I really appreciate all the work you have done. And also the work on new sensors.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on October 29, 2016, 10:21:25 PM
Hi mvdarend,
I prepare the update from my serial gateway from 1.4 to 2.0 (to try your app).
Perhaps you already done on your side ... Did you use the sample code "SerialGateway" ?
(the code seams very light than 1.4 ...)
Bye
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on October 30, 2016, 09:23:14 AM
Perhaps you already done on your side ... Did you use the sample code "SerialGateway" ?

Yes, I started with the Sample code from serial gateway.

(the code seams very light than 1.4 ...)

The 1.4 version written by Haitch is heavily commented and also written in Python which makes the source file larger than my version. (I'm not too good at commenting my code as well as he did)
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 03, 2016, 10:19:32 PM
Hi,
I successfully update my serial gateway and my devices still communicate with the gateway (node still in v 1.4), youou !
Now i try again your app, it said "Connected" so that sound good.
But none of my 2 devices appears (teleingo and RGBW) in the module list "HomeAutomation.MySensors" ...
I start again this device to be discover again but no more chance .. Perhaps because not really in your list of compatibility ...
Bye
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on November 04, 2016, 09:08:09 AM
You say the nodes are still running MySensors v1.4? How are you Presenting these nodes? RGB and RGBW didn't exist until version v1.5.4 (and I have no idea what teleingo is)

Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 04, 2016, 10:29:46 AM
You say the nodes are still running MySensors v1.4?
Yes that right but with my other homeautomation system (dedicated to MySensors), my two devices work still fine.
How are you Presenting these nodes?
What do you means ? I can probably send the log which can permit to see that...
RGB and RGBW didn't exist until version v1.5.4 (and I have no idea what teleingo is)
On my other system, I can use the RGBW by setting value to each color between 0 to 100. So perhaps not recognized as a really RGBW => I will try to update the node too.
Teleinfo (not teleingo) is a French system that permit to plug a mode to the electricity counter to have the electric consumption.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on November 04, 2016, 10:49:43 AM
In your node sketch you should have a 'Presentation' line that looks like this:

Code: [Select]
gw.present(Sensor_Id, Sensor_Type);
In version 2.0 you would have S_RGB_LIGHT or S_RGBW_LIGHT for Sensor type, but your node version is 1.4 and RGB and RGBW didn't exist then. I'm curious as to how you're presenting the RGB node (and also the teleinfo node)

But I have to say that I won't be able to support this APP very well in the future, I have decided to use another Home automation system and no longer have HG running.

Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 04, 2016, 10:51:43 PM
In your node sketch you should have a 'Presentation' line that looks like this:
Code: [Select]
gw.present(Sensor_Id, Sensor_Type);In version 2.0 you would have S_RGB_LIGHT or S_RGBW_LIGHT for Sensor type, but your node version is 1.4 and RGB and RGBW didn't exist then. I'm curious as to how you're presenting the RGB node (and also the teleinfo node)
RGBW present like this (mysensor 1.5 instead 1.4):
Code: [Select]
0;255;3;0;9;TSP:MSG:READ 4-4-0 s=255,c=3,t=11,pt=0,l=4,sg=0:RGBW
4;255;3;0;11;RGBW
And teleinfo like this (mysensor 1.5.4 instead 1.4):
Code: [Select]
0;255;3;0;9;TSP:MSG:READ 1-1-0 s=1,c=1,t=25,pt=0,l=4,sg=0:BASE
1;1;1;0;25;BASE
0;255;3;0;9;TSP:MSG:READ 1-1-0 s=3,c=1,t=26,pt=0,l=4,sg=0:TH..
1;3;1;0;26;TH..
But I have to say that I won't be able to support this APP very well in the future, I have decided to use another Home automation system and no longer have HG running.
So your test on HomeAssistant convince you ?
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on November 05, 2016, 06:52:48 AM
I'm not great at interpreting MySensor messages, but it looks like what you've posted above does not contain a Presentation message.

Here is a breakup of what a message is made up of, it's for version 2.0 but it should be backwards compatible.
https://www.mysensors.org/download/serial_api_20 (https://www.mysensors.org/download/serial_api_20)

As you can see the third value is MessageType, for a message to be a 'Presentation message' the third value must be '0'. In your examples, the values are '3' (Internal) and '1' (Set). The MySensor App that I wrote will ignore these messages as no corresponding module has been created.

Do you have the actual Arduino sketches you wrote?

Sidenote: The MQTT version of MySensors that cptJack created didn't support 'Presentation', this was a limitation of the MQTT gateway itself, in his code a new module was created if an unknown node sent a value.

Quote
So your test on HomeAssistant convince you ?
Yes, I'm fully running on HA now. A lot of things in HomeAssistant are not as good as HomeGenie, but it does what I need and it has a slightly better WAF.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 05, 2016, 06:21:22 PM
Hi mvdarend,
I successfully add the teleinfo node (detected as "sensor"), but RGW is identify as "4" -> Nothing in your code :
Code: [Select]
4;3;0;0;4;
0;255;3;0;9;TSP:MSG:READ 4-4-0 s=5,c=0,t=4,pt=0,l=0,sg=0:
4;5;0;0;4;
0;255;3;0;9;TSP:MSG:READ 4-4-0 s=6,c=0,t=4,pt=0,l=0,sg=0:
4;6;0;0;4;
0;255;3;0;9;TSP:MSG:READ 4-4-0 s=9,c=0,t=4,pt=0,l=0,sg=0:
4;9;0;0;4;
So i add in your code the case "4" :
Code: [Select]
       
case 4: // ???
        hgType = "Dimmer";         
        hgWidget = "homegenie/generic/dimmer";
        parameterValue = "S_DIMMER";
        break;
i try with RGB too :
Code: [Select]
 
case 4: // ???
        hgType = "Dimmer";
        hgWidget = "homegenie/generic/colorlight";
        parameterValue = "S_RGB_LIGHT";
        break;
But it doesn't work fine, I have now device in HomeGenie but I can't command my rgbw...Did you have any idea ? Perhaps, i need to have a review of my RGB sketch ...
Bye
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on November 06, 2016, 07:10:34 AM
Your first option was correct, 4 is for S_DIMMER. Because it is an actuator you can best uncheck 'Group Subdevices with ID < 10' which will then make a separate  module/widgte for each dimmer you are presenting.

I hadn't fully built in support for dimmers (as I didn't have one at the time) but a start has been made, at line 350 you'll see the following:
Code: [Select]
var module = Modules.InDomain(_HG_DOMAIN).WithAddress(reqs[1]).Get();
Once HG has got the module it tries to determine what it it, and the follwing line should fire if the module is a dimmer:

Code: [Select]
      else if (reqs[2] == "Control.Level")
      {
        subType = 3; // V_PERCENTAGE, V_DIMMER
        setValue = reqs[3].ToString();
      }

It's possible that the value set in setValue is not correct. (For example, HG might be sending 0.5 while the MySensor node expects the value 50) Set a debug line there to see what you're getting from HG, eg:

Code: [Select]
      else if (reqs[2] == "Control.Level")
      {
        Program.Notify("Control.Level from HG", reqs[3].ToString()); Pause(1);
        subType = 3; // V_PERCENTAGE, V_DIMMER
        setValue = reqs[3].ToString();
      }

That should point you in the right direction.

Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 06, 2016, 05:03:32 PM
You point me in the good direction  :) Thanks a lot !

I succeed switch on/off my several color, by doing this modifications :

--> Add my type :

Code: [Select]
       
case 4: // V_DIMMER, V_PERCENTAGE
        hgSubType = "Status.Level"; // status.percentage?
        break;

--> Update severals value by * 100 :

Code: [Select]
setValue = "100"; /////
Code: [Select]
{setValue = "100";}
Code: [Select]
setValue = (100 * Convert.ToInt32(reqs[3])).ToString();
--> Changing the send of the command to use serial instead network :

Code: [Select]
TcpClient.SendMessage(theCommand);Replace by
Code: [Select]
SerialPort.SendMessage(theCommand);
--> Changing the send of the command to use serial instead network :

Replace this :
Code: [Select]
subType = 2; //V_STATUS, V_LIGHTby this :
Code: [Select]
subType = 3; //V_DIMMER
########################################
The ToDo to this program :
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: mvdarend on November 06, 2016, 08:14:25 PM
Quote
Changing the send of the command to use serial instead network :
Oops, you've found a bug... It should have looked at the DeviceType variable before senidng, something like this:
Code: [Select]
    // send the message
    switch(DeviceType){
      case  "ethernet":
        TcpClient.SendMessage(theCommand);
        break;
      case "serial":
        SerialPort.SendMessage(theCommand);
        break;
     }

If you (or anyone else) would like to take ownership of the code, please be my guest. I can no longer test any changes so I have no idea if it will work or not.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: raptorjr on November 07, 2016, 10:13:21 AM
You point me in the good direction  :) Thanks a lot !

I succeed switch on/off my several color, by doing this modifications :

--> Add my type :

Code: [Select]
       
case 4: // V_DIMMER, V_PERCENTAGE
        hgSubType = "Status.Level"; // status.percentage?
        break;

--> Update severals value by * 100 :

Code: [Select]
setValue = "100"; /////
Code: [Select]
{setValue = "100";}
Code: [Select]
setValue = (100 * Convert.ToInt32(reqs[3])).ToString();
--> Changing the send of the command to use serial instead network :

Code: [Select]
TcpClient.SendMessage(theCommand);Replace by
Code: [Select]
SerialPort.SendMessage(theCommand);
--> Changing the send of the command to use serial instead network :

Replace this :
Code: [Select]
subType = 2; //V_STATUS, V_LIGHTby this :
Code: [Select]
subType = 3; //V_DIMMER
########################################
The ToDo to this program :
  • Status update of the level in HG
  • Manage set command by serial or network gateway
  • Manage SubType more global

I don't know what the standard for dimmer values are, if all/most dimmers use 0-1 or 0-100? But otherwise maybe better to change your sketch to handle the conversion?
So that more users can use your updated program, and not everyone need to change the values for their usage.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 07, 2016, 10:53:09 PM
raptorjr, my MySensors script do something like this :
Code: [Select]
255 * gw.loadState(RED) / 100But it's a generic sketch ...

I do the change in the following script but if in your case : "switch", it doesn't work. I can manage the 2 case.
Can you test with this new one please?
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: raptorjr on November 07, 2016, 11:25:04 PM
raptorjr, my MySensors script do something like this :
Code: [Select]
255 * gw.loadState(RED) / 100But it's a generic sketch ...

I do the change in the following script but if in your case : "switch", it doesn't work. I can manage the 2 case.
Can you test with this new one please?

I don't have a dimmer, sorry if I was unclear. Just thought that since you seem to know a little about programming, and mvdarend is leaving HG, it would be nice if your changes was generic so everyone could use the things you added.

But as I said, don't know what ranges that are most common, 0-1 or 0-99?

I will use the MySensors gateway a lot, but at the moment I'm about to move, so all fun electronic projects have been put on hold for a while. But I hope that when the time comes that I can help to debug and test the gateway, and maybe come up with suggestions and add code also.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: raptorjr on November 08, 2016, 08:30:13 PM
@nolio

I'm no expert, but when you use f.ex. setValue = "100" for Control.On and Control.Toggle, it feels like your hardcoding things to work with your dimmer? And it wont work with normal light switches or relays? Maybe better to adjust your sketch? 0 and 1 seems more logical for On/Off than 0/100.

I don't know if you are interested in getting a working MySensors gateway or if you just are modifying things to work in your environment? Last option is fine. I'm just looking for someone with the right knowledge to take over after mvdarend =) I'm not at that level yet, but would like to be in the future. But in the meantime we need someone with other MySensor nodes and some developer skills that could help and improve the gateway.

I only have one node with temp, relay and waterflow sensor. So that is the only things I can test at the moment.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 08, 2016, 11:11:26 PM
I'm no expert, but when you use f.ex. setValue = "100" for Control.On and Control.Toggle, it feels like your hardcoding things to work with your dimmer? And it wont work with normal light switches or relays? Maybe better to adjust your sketch? 0 and 1 seems more logical for On/Off than 0/100.
I think my method is the good way and no one seams test that ... But perhaps, i do a mistake. So if someone test and comeback with a problem, i can correct that.
I don't know if you are interested in getting a working MySensors gateway or if you just are modifying things to work in your environment? Last option is fine. I'm just looking for someone with the right knowledge to take over after mvdarend =) I'm not at that level yet, but would like to be in the future. But in the meantime we need someone with other MySensor nodes and some developer skills that could help and improve the gateway.
I am interest for the both ;) but i am ok with you : First (work for everybody). I already post some HG program on other point in this target :)
I only have one node with temp, relay and waterflow sensor. So that is the only things I can test at the moment.
Ok that's a good point to know. On my side
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: raptorjr on November 08, 2016, 11:37:41 PM
I'm no expert, but when you use f.ex. setValue = "100" for Control.On and Control.Toggle, it feels like your hardcoding things to work with your dimmer? And it wont work with normal light switches or relays? Maybe better to adjust your sketch? 0 and 1 seems more logical for On/Off than 0/100.
I think my method is the good way and no one seams test that ... But perhaps, i do a mistake. So if someone test and comeback with a problem, i can correct that.

If you look at the MySensors protocol https://www.mysensors.org/download/serial_api_20 (https://www.mysensors.org/download/serial_api_20) they use 0/1 for On/Off for V_STATUS, V_LIGHT. And if this should be some kind of official MySensor gateway for HG maybe it would be smart and try to follow the protocol? Maybe less problem in the future if more user would like to test MySensors and HG.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 09, 2016, 09:39:21 PM
I add the compatibility to the 2 set command :
by this type of code :
Code: [Select]
         
if (module.IsOfDeviceType("Switch")) {setValue = "1";} //V_STATUS, V_LIGHT
if (module.IsOfDeviceType("Dimmer")) {setValue = "100";} //V_DIMMER
So it's more adaptable :)

Program pull request on github : https://github.com/genielabs/homegenie-packages
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: raptorjr on November 09, 2016, 11:02:45 PM
Yes, would be nice with this included as a package.

But there is another place where SendMessage is used (line 296) that only use TcpClient and not the SerialPort. Suggest that the switch for Tcp or Serial Send message is moved to it's own function so it is easier to use anywhere in the code and no need to do the switch statement every time.
Title: Re: MySensors v2.0 Serial/Ethernet Gateway
Post by: nolio on November 10, 2016, 09:12:12 PM
Funny, I think about that too :)

So i submit with this modification :
https://github.com/genielabs/homegenie-packages/tree/master/packages/Devices%20and%20Things
From
https://github.com/noliono/homegenie-packages/tree/master/packages/Devices%20and%20Things/MySensors