Help Parse Web JSON data

[xefil]

Hello,

I've installed a little HomeGenie server and would like to move my actual monitoring structure into HomeGenie.
Actually I'm using some Arduinos in a meshed netowork (radio) to comunicate each other sensors data or send commands.
I would migrate this infrastructure without touching what already done. The central node expose a json file via web with the informations collected from the slaves.
To get temperature I need to contact the central node at a specificn URL. Then, parsing the json file I can identify which temperature is taken (from which room).
How can I:

• Get this data and show it on a Widget?
• Get this data and graph it on a time basis?
• Import old data I've already collected?

The next step will be use the same technique to send data to the same arduino to trigger some actions.

Thanks a lot for the help!

Simon

[mvdarend]

Parsing JSON data is very easy in HomeGenie, take a look at the Weather Underground or OpenWeatherData APPs.

Another simple example is my Youless Energy monitor APP.

[xefil]

Parsing JSON data is very easy in HomeGenie, take a look at the Weather Underground or OpenWeatherData APPs.

Another simple example is my Youless Energy monitor APP.

Thank you for the answer!
I'm starting to create a new program starting from the OpenWeatherData as example. I've written the following hgx file. It will be imported correctly, but trying to check the values I should supply (input), the tab is empty.
I expect to be able to add these values after enabling the program: url, LeafNodeId01, LeafNodeId01Enabled, LeafNodeId02, LeafNodeId02Enabled...
But the "OPTIONS" tab is empty, seems broken, OPTIONS totally missing.
What's wrong?

The Widget (Program.AddControlWidget("xefil/xefilhomearduino/xefilhomearduino") actually doesn't exist but should in any case show me the popup to enter the data, or not?

HEre my code:

Code: [Select]

<?xml version="1.0" encoding="utf-16"?>
<ProgramBlock xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <ConditionType>OnTrue</ConditionType>
  <Conditions />
  <Commands />
  <ScriptCondition>Program.Setup( () =&gt; {
  // list the program as custom weather widget
  Program.AddControlWidget("xefil/xefilhomearduino/xefilhomearduino");

  // set input fields parameters
  Program.AddInputField("URL", "http://192.168.1.1/test.xml", "Arduino 00 Leaf Station URL");
  Program.AddInputField("LeafNodeId01", "0", "Arduino 1st Leaf");
  Program.AddInputField("LeafNodeId01Enabled", "TRUE", "Arduino 1st Leaf Enabled? Set TRUE for YES, FALSE for NO");
  Program.AddInputField("LeafNodeId02", "2", "Arduino 2nd Leaf");
  Program.AddInputField("LeafNodeId02Enabled", "TRUE", "Arduino 2nd Leaf Enabled? Set TRUE for YES, FALSE for NO");
 

  // setup program parameters
  // Leaf 01
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Node").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Cmd").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasDHT11").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT11_H").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT11_T").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasRelay").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay01").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay02").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay03").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay04").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasDHT22").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT22_H").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT22_T").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasOneWire").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Dallas01").Value = "";
 
  // Leaf 02
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Node").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Cmd").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasDHT11").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT11_H").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT11_T").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasRelay").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay01").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay02").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay03").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay04").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasDHT22").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_H").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_T").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasOneWire").Value = "";
  Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Dallas01").Value = "";
         
  Program.Parameter("xefil.XefilHomeArduino.LastUpdated").Value = "";
});

return true;
</ScriptCondition>
  <ScriptSource>

Func&lt;bool&gt; _checkWeather = new Func&lt;bool&gt;(()=&gt;{
  Program.RunAsyncTask(()=&gt;
{
  string webserviceurl = Program.InputField("URL").Value;
     
      TimeSpan t = (DateTime.UtcNow - new DateTime(1970, 1, 1));
      int UTCTimeNow = (int)t.TotalSeconds;
      string UCTTimeNowString = UTCTimeNow.ToString();
     
      Program.Parameter("xefil.XefilHomeArduino.LastUpdated").Value = UCTTimeNowString; //last_updated;
     
      try
      {
 
        // get data
        var data = Net.WebService(webserviceurl).GetData();   
     
        //check for update in data     
        //if(data.dt.ToString() != Program.Parameter("jkUtils.OpenWeatherMap.Dt").Value ) {
        if(0 != 1 ) {

Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Node").Value = data.nodes.node ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Cmd").Value = data.nodes.cmd ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasDHT11").Value = data.nodes.has_dht11 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT11_H").Value = data.nodes.DHT11_H ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT11_T").Value = data.nodes.DHT11_T ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasRelay").Value = data.nodes.has_relay ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay01").Value = data.nodes.relay01 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay02").Value = data.nodes.relay02 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay03").Value = data.nodes.relay03 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Relay04").Value = data.nodes.relay04 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasDHT22").Value = data.nodes.has_dht22 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT22_H").Value = data.nodes.DHT22_H ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.DHT22_T").Value = data.nodes.DHT22_T ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.HasOneWire").Value = data.nodes.has_onewire ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId01.Dallas01").Value = data.nodes.dallas01 ?? "";

Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Node").Value = data.nodes.node ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Cmd").Value = data.nodes.cmd ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasDHT11").Value = data.nodes.has_dht11 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT11_H").Value = data.nodes.DHT11_H ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT11_T").Value = data.nodes.DHT11_T ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasRelay").Value = data.nodes.has_relay ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay01").Value = data.nodes.relay01 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay02").Value = data.nodes.relay02 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay03").Value = data.nodes.relay03 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Relay04").Value = data.nodes.relay04 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasDHT22").Value = data.nodes.has_dht22 ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_H").Value = data.nodes.DHT22_H ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_T").Value = data.nodes.DHT22_T ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.HasOneWire").Value = data.nodes.has_onewire ?? "";
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.Dallas01").Value = data.nodes.dallas01 ?? "";
   
         
        }
       
        Program.Notify("xefil: XefilHomeArduino","Data has been updated.");
       
      }
      catch (Exception e)
      {               
        Program.Notify("xefil: XefilHomeArduino", "Update of data failed.");
        DEBUG: Program.Notify("XefilHomeArduino ERROR!", e.Message);
        Console.WriteLine( e.Message );
        //Pause(10);       
      }
 
     
     
    });
 
  return true;
});     

//
// web service calls handling
//
When.WebServiceCallReceived("HomeAutomation.HomeGenie.Automation", (args) =&gt; //"HomeGenie.SecuritySystem", ( args ) =&gt;
{
    string[] reqs = ((string)args).Split('/');
    var res = "";
   
    try
    {
      string command = reqs[2];
      string pid = reqs[1];
      if (pid == Program.Module.Address)
      {

        switch(command)
        {
          case "Control.On":

          break;
          case "Control.Off":

            break;
          case "Control.Refresh":
            //Program.Notify("jkutils: OpenWeatherMap","Update triggered.");
          _checkWeather();
            break;
        }

      }
    }
    catch (Exception ex)
    {
      res = "{ 'ResponseValue' : 'ERROR: "   ex.Message   " "   ex.StackTrace   "' }";
    }
    // unable to process request
    return res;

});

while (Program.IsEnabled)
{
  _checkWeather();
  Pause(Program.InputField("UpdateInterval").DecimalValue * 60); // pause before next check
  //Pause(10);
}

</ScriptSource>
  <ScriptErrors />
  <IsRunning>true</IsRunning>
  <Features />
  <LastConditionEvaluationResult>true</LastConditionEvaluationResult>
  <Domain>HomeAutomation.HomeGenie.Automation</Domain>
  <Address>1979</Address>
  <Name>xefil - XefilHomeArduino</Name>
  <Description>Gathers data from XefilHomeArduino</Description>
  <Group>Contrib</Group>
  <ActivationTime>2014-12-29T15:00:00.000000Z</ActivationTime>
  <TriggerTime>2014-12-29T15:00:00.000000Z</TriggerTime>
  <Type>CSharp</Type>
  <IsEnabled>true</IsEnabled>
</ProgramBlock>

Thanks for the help!

Simon

[Gene]

Hi xefil,

I tried the code you posted and it's working as expected.
Did you compile the program?

Cheers,
g.

[xefil]

Hi xefil,

I tried the code you posted and it's working as expected.
Did you compile the program?

Cheers,
g.

oh sorry, how should i compile it?

[Gene]

Like by clicking the "Compile" button?!?!? =)

Cheers,
g.

[xefil]

Like by clicking the "Compile" button?!?!? =)

Cheers,
g.

Uhm sorry Gene. I was on an old HG version (411) and the Compile button was missing. I've used the update ACTION on the program after reading your message. Then, updating the software, I've got the correct button.
I'm getting an error and noticing a strange behaviour:
Trying to update the program, I get "CR: Thread creation failed.", seems on line:

Code: [Select]

Func<bool> _checkXefilXML = new Func<bool>(()=>{
How debug?

The whole HGX file is attached.

And then the system seems hangs. I need to restart the software multiple times until the program appears disabled.

Thanks for help!

Simon

[Gene]

What system/configuration are you running homegenie on?
Also try avoiding the use of "Program.RunAsyncTask".

Cheers,
g.

[xefil]

What system/configuration are you running homegenie on?
Also try avoiding the use of "Program.RunAsyncTask".

Cheers,
g.

I'm using HG on a BananaPI board (armhf Bananian/Raspbian OK).
I would avoit using "Program.RunAsyncTask" but what use instead?
Really I'm using this program to parse a JSON file so I've tried the example from the 'OpenWeatherData' as suggested from mvdarend. Every alternative is OK for me.
The prupose is initially parse an JSON URL content from an Arduino shield. After that I would:
- send HTTP commands to the same Arduino
- graph some values.
But I would start to make one step at time, so first I need to be able to parse the file content. The output from the URL is something like this:

Code: [Select]

{
"nodes":[
{
"node":0,
"cmd":0,
"has_dht11":1,
"DHT11_H":3600,
"DHT11_T":2500,
"has_relay":0,
"relay01":0,
"relay02":0,
"relay03":0,
"relay04":0,
"has_dht22":0,
"DHT22_H":0,
"DHT22_T":0,
"has_onewire":0,
"dallas01":0
},
{
"node":2,
"cmd":0,
"has_dht11":1,
"DHT11_H":4400,
"DHT11_T":1600,
"has_relay":0,
"relay01":0,
"relay02":0,
"relay03":0,
"relay04":0,
"has_dht22":1,
"DHT22_H":2570,
"DHT22_T":140,
"has_onewire":1,
"dallas01":1706
}
]
}

I don't pretend to have the program written by you  (even if a help is always appreciated)
I would try by myself so I'm started from this APP.

Thanks for the help!

Simon

[mvdarend]

Here's a post of mine which gives an example of data I'm trying to parse and how it works:

http://www.homegenie.it/forum/index.php?topic=429.0

You're JSON is in an array, so you'll also need to take that into account.

[xefil]

Hello!

Could you share with me the code?
The issue seems related to "Func<bool> _checkXefilXML = new Func<bool>(()=>{" (copied from the other example).
In addition Gene suggest not to use RunAsyncTask. So, an example as you've it done could help me

Simon

[mvdarend]

You're probably better off using my YouLess Energy Monitor code as it's a lot less complex than the weather examples, I've added an example of the JSON data to the thread so you can compare the code to the data being queried.

If you import the Youless APP and replace all the code in the try block with this it should work. (I haven't tested this though, and I've not queried a JSON array before, so you'll probably have to adapt it a little.)

Code: [Select]

       var myData = Net.WebService("http://192.168.0.100/myJSONPage/").GetData();
       
       // Query the first node, don't know if this is the correct syntax though...
       int DHT22_T = int.Parse(myData.nodes[0].DHT22_T.ToString());
       int DHT22_H = int.Parse(myData.nodes[0].DHT22_H.ToString());
       
       
       Program.Parameter("Sensor.Humidity").Value = DHT22_T; // Should value be divided by 100?
       Program.Parameter("Sensor.Temperature").Value = DHT22_H; // Should value be divided by 100?

[xefil]

You're probably better off using my YouLess Energy Monitor code as it's a lot less complex than the weather examples, I've added an example of the JSON data to the thread so you can compare the code to the data being queried.

If you import the Youless APP and replace all the code in the try block with this it should work. (I haven't tested this though, and I've not queried a JSON array before, so you'll probably have to adapt it a little.)

Code: [Select]

       var myData = Net.WebService("http://192.168.0.100/myJSONPage/").GetData();
       
       // Query the first node, don't know if this is the correct syntax though...
       int DHT22_T = int.Parse(myData.nodes[0].DHT22_T.ToString());
       int DHT22_H = int.Parse(myData.nodes[0].DHT22_H.ToString());
       
       
       Program.Parameter("Sensor.Humidity").Value = DHT22_T; // Should value be divided by 100?
       Program.Parameter("Sensor.Temperature").Value = DHT22_H; // Should value be divided by 100?

Thanks for the suggestions!
Seems it starts to work!
I've a little issue here:

Code: [Select]

int DHT22_T = int.Parse(data.nodes[1].DHT22_T.ToString());
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_T").Value = (DHT22_T / 100).ToString();

The sensor reads: -120 (yess, it needs to be divided by 100).
After adding the value to the string, I only can read "-1" from the details.

Then, if I leave the "catch (Exception e) " to log in case of errors I get:

- Unexpected symbol `catch'
- Unexpected symbol `e'
- Unexpected symbol `)'

Code: [Select]

     catch (Exception e)
     {
        Program.Notify("xefil: XefilHomeArduino!", "Error retrieving XefilHomeArduino data!");
        //Program.Parameter("YouLess.Current").Value = "ERROR: " + e.Message;
        Console.WriteLine( e.Message );
        Pause(5);
     }


[mvdarend]

The first problem sounds like a rounding error, you could try something line this:

Code: [Select]

int DHT22_T = int.Parse(data.nodes[1].DHT22_T.ToString());
double temperature = DHT22_T / 100

Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_T").Value = temperature.ToString();

The catch needs to be part of the try.. catch block, like this

Code: [Select]

try
{
  // your code here
}
catch (Exception e)
{
  Program.Notify("xefil: XefilHomeArduino!", "Error retrieving XefilHomeArduino data!");
  //Program.Parameter("YouLess.Current").Value = "ERROR: " + e.Message;
  Console.WriteLine( e.Message );
  Pause(5);
}

* I haven't tested any of the above code, so it may not work out of the box, but it should help you further

[xefil]

The first problem sounds like a rounding error, you could try something line this:

Code: [Select]

int DHT22_T = int.Parse(data.nodes[1].DHT22_T.ToString());
double temperature = DHT22_T / 100

Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_T").Value = temperature.ToString();


This is not working but you've pointed me to the right direction. I've solved with:

Code: [Select]

float DHT22_T = (float.Parse(data.nodes[1].DHT22_T.ToString()) / 100);
Program.Parameter("xefil.XefilHomeArduino.LeafNodeId02.DHT22_T").Value = DHT22_T.ToString();


The catch needs to be part of the try.. catch block, like this

Code: [Select]

try
{
  // your code here
}
catch (Exception e)
{
  Program.Notify("xefil: XefilHomeArduino!", "Error retrieving XefilHomeArduino data!");
  //Program.Parameter("YouLess.Current").Value = "ERROR: " + e.Message;
  Console.WriteLine( e.Message );
  Pause(5);
}

* I haven't tested any of the above code, so it may not work out of the box, but it should help you further

Thank you for this, that worked well!

So, now I can complete to parse the whole array.
Meanwhile, I've another question (in real I've a lot of questions, but one step per time )
If I would like to graph this values, should they saved in a different way? I've seen in your example it uses 'Meter.Total' and 'Meter.Watts'. Has it to do something with that?

I see they should selectable on "Analysis" page.

Thanks again!

Simon

PS: After that I'll try to create the WIDGET