Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

External sensors #11

Open
robross0606 opened this issue Nov 27, 2019 · 109 comments
Open

External sensors #11

robross0606 opened this issue Nov 27, 2019 · 109 comments

Comments

@robross0606
Copy link

robross0606 commented Nov 27, 2019

Most of the instructions and design focus on exposing ST devices to external systems. However, I would like to introduce external sensors (motion, open/close, etc.) into ST. It is unclear to me how I could create a device in SmartThings that equates to a device that originated in HomeAssistant. I've seen some discussion of using virtual switches for simple on/off switches, but how can this be done for sensors?

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

If you have device / sensors in Hass I believe u don’t even have to create a config - this is the legacy implementation that I continue to support.

So just make sure the device capability u are trying to import is there in the SmartApp capability map and make sure u have a virtual device to actually interface with it

Since I don’t have a HASSio environment I have not been able to test but perhaps you can.

So look at the config file and the big section that is all commented out.
I believe HA published MQTT topic in that format preface/device/attribute/command and the server automatically configure that topic. ( however if you specific the same device in the config file - that will override this logic and use what u specify in config file)

@robross0606
Copy link
Author

robross0606 commented Nov 28, 2019

So, as an example, I have a motion sensor which is native to HA (not ST). I want that in ST. I already created a device in ST using "motion.capability" as the Type. Then I opened the MBS SmartApp on my mobile device and choose that newly created "virtual" device under the "Motion" section of the MBS SmartApp (which aligns to "motion.capability" in your CAPABILITY_MAP).

What I don't understand is how to get the devices aligned and mapped to MQTT topics. I must be missing it because I don't see where you set up the topic structure. I have the node.js service up and running but don't see any topics being published for native HA devices. I had assumed I needed to set up my device.cfg file on the node.js service.

BTW, I don't have a HASSio environment either. I'm running HA on a Windows machine right now.

@robross0606
Copy link
Author

robross0606 commented Nov 28, 2019

What's really bizarre is that I took a look at the logs on the node.js service and saw this:

2019-11-28 09:05:41 AM info: ===================================ACTUAL SUBSCRIPTIONS REQUESTED FROM SMARTAPP ============================================
2019-11-28 09:05:41 AM info: Currently subscribed to smartthings/KBR-Door/contact, smartthings/Electricity Meter/demand, smartthings/Electricity Meter/mqttmsg, smartthings/Contacts/notify, smartthings/System/notify, smartthings/LR-Recessed-M3/switch, smartthings/Patio-M2/switch, smartthings/LR-FanLight-M4/switch, smartthings/LR-Fan-M5/switch, smartthings/PoolMotor-M6/switch, smartthings/OutdoorLight-M7/switch, smartthings/SNF2/switch, smartthings/SNF3/switch, smartthings/Tasmota Test/switch, smartthings/KBR-Light-M8/switch, smartthings/MBR-FanLight-M9/switch, smartthings/a-wemos-ch1/switch, smartthings/a-wemos-ch2/switch, smartthings/GBR-Fan-M10/switch, smartthings/GBR-FanLight-M10/switch, smartthings/LR-Lamp1/switch, smartthings/LR-Lamp2/switch, smartthings/KBR-Heater/switch, smartthings/LR-Recessed-M3/update, smartthings/Patio-M2/update, smartthings/LR-FanLight-M4/update, smartthings/LR-Fan-M5/update, smartthings/PoolMotor-M6/update, smartthings/OutdoorLight-M7/update, smartthings/SNF2/update, smartthings/SNF3/update, smartthings/Tasmota Test/update, smartthings/KBR-Light-M8/update, smartthings/MBR-FanLight-M9/update, smartthings/a-wemos-ch1/update, smartthings/a-wemos-ch2/update, smartthings/GBR-Fan-M10/update, smartthings/GBR-FanLight-M10/update
2019-11-28 09:05:41 AM info: ============================================================================================================================

I took another look at the MBS SmartApp code and there are ZERO references to these devices. Just to be safe, not only did I disable the separate devices.cfg file but I physically removed it from the config folder. From where is it getting this information?

@robross0606
Copy link
Author

Okay, figured that part out. It was using old data in the "state" folder. I had to delete those files to remove it. So, with that resolved, the file now shows that it is subscribing to zero things. I don't see any publish activity at all either so how do I get the node.js service to publish native HA devices?

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

Okay u can do it two ways

First
Can u pls provide the topic format for subs and pubs that these native hass devices r publishing?
Is it the same format as the commented out section under MQTT config in the config.yml?
If yes u just need to uncomment those parts out, specify the right preface and should be all good.

2nd part
I recommend this as more flexibility.
In config.yml set deviceconfig flag to true.

Create a devices.yml file - follow the format as in the devices.example file - that is where u would specify the topics and commands

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

How did u install this ? Did u use docker image or NPM? I must have messed up somewhere in an upload - didn’t mean to upload my config files with log and state information - please make sure all directories under config are empty and only populated with your config.yml and devices.yml. I need to fix whatever source you downloaded these from - please let me know

@robross0606
Copy link
Author

I installed via directly NPM. I think it was my fault. Your files come down with a devices.cfg file that contains your setup. I must've run once with that accidentally before removing it.

@robross0606
Copy link
Author

One other question I have is that you say you need to enter a MAC address but don't give an example to derive format. MAC addresses can be represented many different ways, so I don't know how to format the hex values.

@robross0606
Copy link
Author

robross0606 commented Nov 28, 2019

Can u pls provide the topic format for subs and pubs that these native hass devices r publishing?

That's what I'm saying. NOTHING is being published to MQTT at all from HASS. All that I see in the log file for your service is this:

2019-11-28 09:14:19 AM info: Loading previous state
2019-11-28 09:14:19 AM info: request object - [object Object]
2019-11-28 09:14:19 AM info: ===================================ACTUAL SUBSCRIPTIONS REQUESTED FROM SMARTAPP ============================================
2019-11-28 09:14:19 AM info: Currently subscribed to 
2019-11-28 09:14:19 AM info: ============================================================================================================================
2019-11-28 09:14:19 AM info: Saving current state
2019-11-28 09:14:19 AM info: Connecting to MQTT at mqtt://xxx.xxx.xxx.xxx
2019-11-28 09:14:19 AM info: Configuring autosave
2019-11-28 09:14:19 AM info: Configuring API
2019-11-28 09:14:19 AM info: Listening at http://localhost:8099

I have logging turned all the way up to debug but I see nothing. The service never publishes a thing to my MQTT server automatically.

@robross0606
Copy link
Author

Honestly, now that I look at this, I don't see how this bridge is related to HASS at all. The flow is between SmartThings and MQTT. There's nothing in the configuration of your service that even knows about HASS. You don't configure a connection address or port for HASS. Unless it assumes it is running on the same server as HASS, I'm not sure how your service would even know about native HASS devices.

@robross0606
Copy link
Author

At this point, I've fallen back on just trying to make sure the typical flow is even working. I've added a bunch of native ST devices to the bridge configuration to see if I can get them to flow into your service and be published to MQTT. So far, nothing. Now I'm thinking this may be some type of networking issue. What is the proper expected format for the MAC address?

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

okay one more try:
if you look at config file

Suffix for the topics that receive state from SmartThings $PREFACE/$DEVICE_NAME/$PROPERTY/$STATE_READ_SUFFIX
    # Your physical device or application should subscribe to this topic to get updated status from SmartThings
    # state_read_suffix: state

so assuming in Hass you subscribe to topic in format $DEVICE_NAME/$PROPERTY/state to get a state from device , server will do this automatically for you - it will get your event from hub and publish to this topic.

similarly for command topics

# Suffix for the command topics $PREFACE/$DEVICE_NAME/$PROPERTY/$COMMAND_SUFFIX
    # command_suffix: cmd

assuming hass send $DEVICE_NAME/$PROPERTY/cmd to mqtt broker, server will take that and trigger appropriate event is ST hub

again i do not have a hass implementation so if hass is not publishing to $DEVICE_NAME/$PROPERTY/cmd or subscribing to $DEVICE_NAME/$PROPERTY/state then go with the other option

@robross0606
Copy link
Author

I'm not trying to deviate from the standard "smartthings" prefix or anything. I was right that my mac address structure wasn't correct. I had it with ":" between but changed to "-" and it started working at least for subscriptions. Nothing at all is publishing though. This application architecture isn't really designed to work from native HA devices to virtual ST devices. HA wouldn't know about the devices until something was published by ST. But nothing will ever be published by ST until a state changes. No state would change on a virtual device so the linkage never gets made.
What I am trying to do right now instead is us an Automation in HA with the native mqtt.publish service to publish states from HA onto the same topic as expected by ST. But that doesn't appear to be working either.

@sgupta999
Copy link
Owner

MAC address should be aa:bb:cc:dd:ee:ff format
looks like your ST is not sending subscription to the server so no communication between ST and server yet.

also looks like you are not using devices.yml - so delete state files, fix bridge param in IDE, go to smartapp config scree and save to resubscribe to everything again

@robross0606
Copy link
Author

I do have subscriptions from ST to HA working now. It seems to be functioning for the typical documented use case. However, the reverse direction where state goes from HA to ST and cmd goes from ST to HA doesn't appear to be supported. It isn't a true bi-directional bridge.

@sgupta999
Copy link
Owner

it does work.
Lets say you have a virtual switch.
When you press on button in virtual switch - the hub will generate an event [device][attribute][command/payload].

smartapp will take that and pass it on to the server. server will publish
[device/attribute/state][payload] to mqtt broker - so assuming hass is subscribed to device/attribute/state to receive events from ST and then turn device on.

similarly ST is subscribed to
device/attribute/cmd and device/attribute/set_state assuming topics that hass publishes to

so true two way

@sgupta999
Copy link
Owner

if that is not your hass setup then just use the devices.yml and there you can set up any config you want

  • multiple ST devices listening to one hass topic
  • one ST device listening to multiple hass topics, or one hass topic
  • multiple st devices publishing to one hass topic
  • one st device publishing to one hass topic or multiple hass topics

so if base case doesn't work use this

@robross0606
Copy link
Author

You just described what I said would not happen. You said I would press the button on the virtual switch to initiate the communications ST->HA. That's not what's happening here. There's no "button" to press, nor would I want to have to press it every time anything rebooted in order to set up the bridge communications. The devices are native to HA and all state changes come from HA, not ST. Because of this, the bridge never knows about the device to begin with because nothing is published to MQTT until a state changes. To resolve this, the MBA Bridge SmartApp should be "priming" MQTT with the initial state of all configured devices when the hub starts up. That way the communications would be set up. But, the way it works now, if you never "prime" the devices to HA by triggering state change FIRST in ST, nothing ever communicates.

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

one by one:

first you need the mac address to either have ':' or nothing, not sure the '-' would work (its an ST internal implementation no clue on that).
mac address is needed to get communication from server to bridge(IDE) - so if that is not working look at that. so if you are getting events from the server to the device your current MAC is probably working. sending from bridge(IDE) to server - ip and port are sufficient

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

if you create a virtual switch - and use the default DTH you will have on / off switches in the app.

every time the server starts up it the mqtt broker publishes last retained message (state) to server which is sent to smartapp and IDE so without doing anything you should get the correct state in the virtual device

its upto you if you want to use virtual device to also control the physical device - but when you do click on the virtual device switch the command is published to the appropriate topic - i am assuming once hass receives a command topic from the mqtt broker then it initiates an action on the physical device.

if the physical device has a direct mqtt interface you can even choose to bypass hass if you want to

@sgupta999
Copy link
Owner

if you could please send me the topics you want ST to subscribe to and publish to I might get a better sense of what you are trying to accomplish.

@robross0606
Copy link
Author

robross0606 commented Nov 28, 2019

if you create a virtual switch - and use the default DTH you will have on / off switches in the app.

These are not virtual switches. They are motion and door sensors.

every time the server starts up it the mqtt broker publishes last retained message (state) to server which is sent to smartapp and IDE so without doing anything you should get the correct state in the virtual device

This is still ignoring the FIRST time. The way the system works now, there must be something that happens on the ST side to initiate the data flow. It isn't possible for HA to initiate the data flow. Also, your suggestion assumes the MQTT server is never restarted. If, for example, we were to have a power outage, I would have to go back into ST and "prime" all these virtual devices to get the data flowing with HA again.

@robross0606
Copy link
Author

if you could please send me the topics you want ST to subscribe to and publish to I might get a better sense of what you are trying to accomplish.

I'm not sure why the specific topics matter. I don't really care what the topics end up being. I care about the data actually flowing. What I have is several devices that are native to HA:

  • Six (6) simple contact sensors. Four (4) "door" and two (2) "window".
  • Two simple (2) motion sensors.

I need state data to flow from HA to ST. That means I need to have similar devices set up in ST that are not real. They're not "simulated" either. They're "virtual" but I don't see any option for "virtual" motion or contact sensors -- just buttons and switches.

I don't need any command data to flow the other direction (ST -> HA) right now.

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

Please ignore last message

You are right for motion and contact sensor no data from ST to hass.

You forget that I save state and every 15 minutes I resubsribe everything again if not subscribed. So you wouldn’t have to reconnect these devices again.

I will test with MQTT broker being down for protracted time - not sure I have tested it rigorously but that’s the expected behavior

So for your use case you only need data to flow from hass to bridge to st

@sgupta999
Copy link
Owner

R u seeing any activity in logs to what topics ST is subscribing

@sgupta999
Copy link
Owner

ST does have simulated contact sensor, not sure but think they might have a simulated motion sensor too otherwise use contact sensor as motion sensor.

@sgupta999
Copy link
Owner

sgupta999 commented Nov 28, 2019

Topics matter because if you are not specifying a devices.yml then the server is making assumptions about what format the topic is in based on the config.yml setting and subscribing to those topics - so if hass is not publishing to those topics then u will not see them - but I think u said u were already getting subscription messages from hass to ST

@rhamblen
Copy link
Collaborator

@robross0606 chuck MQTT-spy on a machine on the network. It should help double check what is being passed. You can use it to both listen and send MQTT packets.

and sandeep beat me to the sensors... there in the library, not the default loads.

@robross0606
Copy link
Author

As far as I know, "Simulated" sensors are not the same as "Virtual" sensors. "Simulated" sensors are triggered by physical tile presses from the app. Hence the "tile" being defined in the code.

@sgupta999
Copy link
Owner

let me check something

@robross0606
Copy link
Author

I think it may have actually come from Hass after all. I was triggering the state change for these tests via the Hass developer console. I think right after I artificially changed it, ADT Pulse was changing it back. A real sensor trigger doesn't seem to be exhibiting the same issue. I'd call this success. Now to try the same thing for a contact sensor...

@sgupta999
Copy link
Owner

yes the first line
Duplicate of last event from device 'Kitchen Motion', ignoring event 'inactive' on attribute 'motion' for device 'Kitchen Motion'

suggested this is from the server - the server logs should confirm that.

@sgupta999
Copy link
Owner

glad it was not a switch or it would be a very bothersome loop (i have done it hence all the dupliate checks)

@robross0606
Copy link
Author

Goodness, this thing is going to fight me every step of the way. The contact sensor appears to be flowing all the way through. I can see it in the logs all the way to ST IDE:

50e3cfa0-8b6a-4760-99c6-e0301448b22b 12:29:07 PM: debug Subscribed to event contact on device [Garage Inner Door, Front Door]
50e3cfa0-8b6a-4760-99c6-e0301448b22b 12:28:33 PM: debug Received device event from bridge: [command:true, name:Garage Inner Door, type:contact, value:closed]

And yet the dang UI isn't updating at all. 🙄

@robross0606
Copy link
Author

What I don't see is any logs under the "Garage Door Sensor" device. Perhaps there's a problem with the handler? I'm using the "Simulated Contact Sensor" without modification.

@sgupta999
Copy link
Owner

I dont see action event being called. are you seeing any erorr messges or is this the last message you see

@robross0606
Copy link
Author

No, that's the last thing I see.

@sgupta999
Copy link
Owner

in the capability map do you have

"contactSensors": [
    name: "Contact Sensor",
    capability: "capability.contactSensor",
    attributes: [
        "contact"
    ],
    action: "actionOpenClosed"
],

and further down action

def actionOnOff(device, attribute, value) {
if (value == "off") {
device.off()
} else if (value == "on") {
device.on()
}
}

@sgupta999
Copy link
Owner

sorry
def actionOpenClosed(device, attribute, value) {
if (value == "open") {
device.open()
} else if (value == "closed") {
device.close()
}
}

@sgupta999
Copy link
Owner

action: "actionOpenClosed" needs to be clearly defined in the capability map otherwise app will not know what method to call

@robross0606
Copy link
Author

robross0606 commented Nov 29, 2019

Ahh, that capability did not have an action defined for some reason. It is like that in the repo.

@sgupta999
Copy link
Owner

that is odd..it should have been there

@sgupta999
Copy link
Owner

sgupta999 commented Nov 29, 2019

did you use the tamotaSensor or the default contactSensor - i see it in the defaul contactSensor in the repo.

for tamostaSensor I defined my own custom action methods for custom DTH

@robross0606
Copy link
Author

@sgupta999
Copy link
Owner

could you please post what you have for capability map there - there are several subtleties with the action attribute there wnat to make sure you get it right

@robross0606
Copy link
Author

robross0606 commented Nov 29, 2019

It is already working once I added the action in there. I didn't modify the smartapp at all. I copied exactly as it is in the repo. The only thing I've added is the action on the contact sensor.

@sgupta999
Copy link
Owner

good catch - i was looking at the lite version

@robross0606
Copy link
Author

The whole process from physical device to registration in ST is pretty slow, but I think most of that is on the Hass (ADT Pulse) side. Better than nothing!

@sgupta999
Copy link
Owner

sgupta999 commented Nov 29, 2019

not sure I understand - as soon as message from hass hits your MQTT broker it should be pretty quick after that less than 1s. are you saying hass takes a long time for posting to broker after physical device triggers?

@robross0606
Copy link
Author

Yes, there's a whole part to this flow that I haven't discussed. The whole point to this is tieing in sensors from an ADT Pulse system (non-Smartthings version). There's an add-on for Hass that is webhooked into ADT Pulse to read sensor states. That part of it is slow. I assume it is polling, but haven't checked the code yet. The process once Hass gets around to publishing is pretty quick.

@sgupta999
Copy link
Owner

sgupta999 commented Nov 29, 2019

I think like the contact sensor several other capabilities are also missing action methods - this is the legacy part - these i think like the contact sensor were just meant to change state and not trigger an event. if you are going to specify a command suffix then action method needs to be defined.

I am glad it worked for you - you are our first direct hass integration using legacy code - glad to know i did not break the legacy implementation

@robross0606
Copy link
Author

Once I know this is working, I'll switch over to the new method and break everything again. 😉

@sgupta999
Copy link
Owner

sgupta999 commented Nov 29, 2019

no the new method is a lot more easier and error checked - i couldn't check anything with the hass part before.

everything in the new part is assumed to be an active command - if you don;t want anything to happen just dont specify the action method.

@sgupta999
Copy link
Owner

i don't know if ADT pulse is using wired or wireless sensor but I had a 25 year old security system in the house just attached the wires from com port in parallel to a wemos d1 and st_anything and instantaneous alerts in ST. if all your wires from the alarm system come to a central box that is the way to go.

@robross0606
Copy link
Author

robross0606 commented Nov 29, 2019

ADT Pulse is an annoying beast. It is a "smart" system that connects to an older security board so I think it is "in the way" of me connecting to the central box. The Pulse hub is already connected to the serial pins on the central box. This is still a monitored system, so I have to be careful how I muck with it for now. Once my contract is up, I will be looking at all sorts of alternatives because Pulse sucks.

@robross0606
Copy link
Author

WTH, it suddenly stopped working again for contact sensors. I touched nothing but starting to see this in the ST IDE Live Log:

50e3cfa0-8b6a-4760-99c6-e0301448b22b 3:51:44 PM: error java.lang.IllegalArgumentException: Command 'close' is not supported by device a33e3a9b-7500-4526-8ba4-b1056193244a of type 'Virtual Contact Sensor'. Supported commands: [open, closed] @line 719 (actionOpenClosed)
50e3cfa0-8b6a-4760-99c6-e0301448b22b 3:51:44 PM: debug Calling action method actionOpenClosed, for attribute contact, for device Front Door with payload closed
50e3cfa0-8b6a-4760-99c6-e0301448b22b 3:51:44 PM: debug Received device event from bridge: [command:true, name:Front Door, type:contact, value:closed]

How on EARTH is the command attempting to send "close" when the payload in the log file confirms the value is "closed"?

@sgupta999
Copy link
Owner

I have done a similar mistake in the past.

the default method in DTH is close even thought attribute value is 'closed'.

Did you overwrite the original method's name from close to closed or are you using a new DTH with a closed method instead of close. just check the device handlers change closed method to close

@robross0606
Copy link
Author

I just figured it out a second ago. The method in the device was close() but the command attribute at the top was "closed". Not sure when or how that changed but it is working now. I'm not touching it again for the rest of the weekend. I swear! 😃

@sgupta999
Copy link
Owner

sgupta999 commented Nov 29, 2019

Welcome to quirks of ST :)
my logs are very extensive for now just to diagnose these kind of errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants