This is the first blog post by Anton Kireyeu. A new contributor to Home Assistant who will focus on exploring and visualizing Home Assistant data.

As we learned in the recent blog post by Fabian, all operational data of your Home Assistant application is stored locally and is available for exploration. Our first steps were querying data with the DB Browser for SQLite, exporting the data extract as a CSV file and graphing in LibreOffice. But what else can be done with this data and what tools are there available?

This post will help you get set up using a few popular data scientist tools to allow you to locally process your data:

•  Pandas: an open source tool for data analysis for Python
•  matplotlib: a Python plotting library
•  Jupyter notebook: application for creation and sharing of documents containing live code, visualizations and explanatory text

One of the graphs created with this tutorial.

TL; DR: Use this Jupyter Notebook to visualize of your data

The history component is tracking everything that is going on within Home Assistant. This means that you have access to all stored information about your home. Our history is not a full-fledged graphical processing and visualization component as you may know from systems and network monitoring tools. The current limitation is that you only can select a day for a visual output of your information and not a period. Also, there is no possibility to drill down on a specific entity.

This blog post will show you ways to export data for reporting, visualization, or further analysis of automation rules.

It’s time for Home Assistant 0.24 and it’s full of new integration for your Home. It contains some structural changes to our history which requires action from your end, so please keep reading.

MapQuest discontinued their free and direct tile access by Monday, July 11, 2016. With CARTO we found a very cool and suitable solution. They allow us to use their tile for the map. Thank you, CARTO.

Roy Hooper did an amazing job migrating the history support from being tied to SQLite to use the ORM SQLAlchemy instead. This means that you can now use any SQL backend for the history. So besides SQLite you can now databases like MySQL or PostgreSQL. However, this does require that you install SQLAlchemy and run a command to migrate your existing history over. We tried to make the process as seamless as possible by introducing a new command line script:

$ pip3 install SQLAlchemy
$ hass --script db_migrator --config /path/to/config

You can omit the --config option if you use the default configuration directory. Run the script with --help to get more options.

• Config: Improved support for storing secrets (@kellerza)
• Sensor: Support for Yahoo! Weather (@pvizeli)
• Add scripts to command line to expose advanced options (@balloob)
• Alarm: SimpliSafe is now supported (@w1ll1am23)
• Core: Switch to SQLAlchemy for the Recorder component (@rhooper)
• Support for Join by Joaoapps added incl. Join Notify (@nkgilley)
• Media Player: Plex will no longer spam the logs if server goes offline (@dale3h)
• Sensor: APCUPSd Sensor now supports names, icons and units (@dale3h)
• Lock: Verisure entities will now use name instead of serial number for entity id (@turbokongen)
• StatsD can now also export attributes (@bah2830)
• Support for KNX added (@usul27)
• Switch: TPLink HS100/HS110 now supported (@GadgetReactor)
• Stability fixes for RFXtrx (@Danielhiversen)
• Tweaks to Z-Wave (@turbokongen)
• Light: Brightness now clamped to 0-255 (@keatontaylor)
• Thermostat: Radiotherm HVAC mode now supported (@danieljkemp)
• Sensor: Google Travel times can now use dynamic locations for start/end (@bah2830)
• Notify: Allow sending photos to Telegram (@pvizeli)
• Frontend: Improve loading times (@balloob)
• Frontend: Fix stream not reconnecting after standby (@balloob)
• Frontend: Wait up to two seconds for new state before resetting toggle after toggling state (@balloob)

Hotfix 0.24.1 - July 21

Quick hot fix after we found a bug in the migrator where it wouldn’t work with a database in a non-standard location. Thanks to @n8henrie and @AlucardZero.

Breaking changes

• Migrating existing databases (see above).
• The APCUPSd Sensor was updated. This will need that you modify your configuration.yaml file.
• Entity IDs of Verisure locks will change. This is a one time change but should improve readability.

Over a year ago I participated in the kickstarter campaign for “CHIP - The World’s First Nine Dollar Computer” by Next Thing Co.. I went for the PocketCHIP because of the idea. Display, built-in storage (thus no need for SD cards), battery-powered, and a keyboard are pretty nice features. Last week a package arrives…

It’s time for Home Assistant 0.23 and it’s full of goodies. It’s also the release that bumps us over a 1000 tests and to 94% test coverage! Also our install issues on the Raspberry Pi and Synology have been resolved.

This release brings support for two new ecosystems: Envisalink and Homematic. We can now also control your TV via HDMI using HDMI-CEC (which works on the Pi!) and another cool feature is the persistent notifications which allow you to add a notification to the frontend till dismissed.

Wink support has been dramatically improved by migrating to the PubNub API. This allows Wink to push changes from their system to Home Assistant. This change came just in time as somehow our Wink integration was causing a lot of requests to their servers. Thanks to Wink for letting us know so we could solve it instead of blocking us.

On the config side, you can now store your passwords in your OS keyring or just in a standalone file. We also got a new service to reload the core config so no reboots needed anymore after changing customize settings!

• Support for Envisalink added (alarm control panel, binary sensor, sensor) (@cinntax)
• Support for Homematic added (binary sensor, light, rollershutter, sensor, switch) (@pvizeli, @danielperna84)
• New HDMI-CEC component (@happyleavesaoc, @lukas-hetzenecker)
• Major rewrite of Wink which now pushes changes to Home Assistant (@w1ll1am23)
• Core: new add reload core config service (@balloob)
• Support for persistent notifications added (@fabaff, @balloob)
• Garage door: Z-Wave support added (@turbokongen)
• Rollershutter: Z-Wave support added (@turbokongen)
• Media Player: Sony Bravia TV now supported (@aparraga)
• Sensor: Fixer.io now supported (@fabaff)
• Garage door: Control any garage door using Raspberry Pi GPIO pins (@kellerza)
• Sensor: OpenExchangeRates support added (@arsaboo)
• Notify: Pushover now supports target device, sound, url and priority (@dale3h)
• Sensor: Netatmo now supports wind, battery and radio signals (@Jypy)
• Log successful and failed login attemps (@fabaff)
• Config: allow extracting account info into OS keyring or separate YAML file (@kellerza)
• Core: add option to not filter out duplicate states per entity (@philipbl)
• HTTP: Follow Mozilla SSL recommendations (@danieljkemp, @AlucardZero)
• Light: Z-Wave colorbulb support added (@armills)
• Core: new elevation config option added (@balloob)
• Sensor: OneWire support extended with support for DS18S20, DS1822, DS1825 and DS28EA00 temperature sensors + support for bus masters which use fuse to mount device tree. (@Ardetus)
• Lock: Vera now supported (@rhooper)
• HTTP: Migrate to CherryPy WSGI server to fix install and runtime problems (@balloob)

Breaking changes

• Homematic thermostat configuration has changed and now depends on the new Homematic component.

Hotfix 0.23.1 - July 2

• Bump PyVera to 0.2.13 to fix traceback and pyvera thread dying related to bug (@rhooper)
• HTTP - SSL: Check for OP_NO_COMPRESSION support before trying to use it (@AlucardZero)
• Wink: Downgraded pubnub to work around pycryptodome conflicts (@w1ll1am23)

FAQ

• elevation: was introduced to the configuration for weather/sunrise data. For existing configurations add the value shown in the warning [homeassistant.config] Incomplete core config. Auto detected elevation: 665 to your configuration.yaml file.

In the past month I was thinking about ways to integrate USB webcams into Home Assistant again. The main reason was that this would give those devices a second life and enable one to benefit from low-cost video surveillance. There are a couple of options available like pygame or SimpleCV but I never finished something. With the Local File camera platform by Landrash and motion you could integrate a local USB webcam with a few very easy steps.

In this blog post I am using a Fedora 24 (will most likely work on other distributions too) installation with Home Assistant 0.22.1 on a Foxconn nT-330i with an old Logitech QuickCam Orbit AF and a Logitech HD Webcam C270. As a start only the Quickcam is used. No multi-camera setup for now.

It’s time for the 0.22 release. This was a pretty rough release cycle and we had to issue two hot fixes for our core improvements. But it seems now that all is good and a lot of people have reported that their installs are faster than ever and the occasional quirks no longer occur.

We are aware that our new web stack has caused issues installing Home Assistant on ARM-based platforms. This sadly includes the Raspberry Pi and Synology NAS systems. We’re working on getting to a better solution. For Raspberry Pi, the All-in-One installer will take care of everything for you. We’re working on updating our standalone Raspberry Pi installation guide.

There are two cool things that I want to highlight in this release. The first is Pandora support. This is based on the CLI player called pianobar. This means that your machine running Home Assistant can be connected to the speakers and provide your house with tunes.

Another cool addition is the local file camera. This seems very basic at first but will allow you to generate a graph with your favorite 3rd party graphing tool and display it on your Home Assistant dashboard. We’re looking forward to see what you can do with this!

• Media Player: Pandora media player now supported (@partofthething)
• Device Tracker: BT Home Hub 5 now supported (@lwis)
• Camera: New local file platform shows any image as camera (@Landrash)
• Add Sonos snapshot and restore services (@dansullivan86)
• Device Tracker: AsusWRT in Access Point mode now supported (@linjef)
• Device Tracker: AsusWRT login using public key now supported (@mtreinish)
• Device Tracker: AsusWRT protocol to use is now auto detected (@persandstrom)
• Camera: Netatmo now supported (@jabesq)
• API documentation added in Swagger.yaml format (@wind-rider)
• Media Player: Cast devices can now be stopped (@michaelarnauts)
• MySensors: IR switch device and service now supported (@MartinHjelmare)
• Bloomsky: Voltage sensor now supported (@arsaboo)
• Sensor: New Plex sensor monitors friends streaming from your Plex server (@nkgilley)
• Component shell command can now use templates to render arguments (@partofthething)
• Rollershutter: Wink is now supported (@philk)
• Alexa: Updated documentation to show how to call scripts and scenes (@acockburn)
• Sensor: SNMP is now supported (@fabaff)
• Sensor: Support for Swiss hydrological data (@fabaff)

Breaking change

• The new Netatmo support caused us to change how Netatmo are configured. It’s now done via it’s own component.

netatmo:
    api_key: API_KEY
    secret_key: SECRET_KEY
    username: username
    password: password

Hotfix 0.22.1 - June 20

• Insteon Hub lights will load again

It’s been already almost two weeks ago that a few of the Home Assistant developers headed towards Portland for PyCon 2016 - the conference about everything Python. We were there to learn all the nifty tricks to make our code better but most of all, to talk Home Automation.

Couple of Home Assistant devs. Left to right: Paulus (@balloob), Alex (@infamy), Ryan (@rmkraus).

On Monday I (Paulus) gave a presentation about Home Assistant to an audience of over 400 people! It was a bit scary at first but after a couple of minutes it went all great including some great questions afterwards. Slides can be found here and the talk is embedded right below:

One of the things that really impressed me was the amount of people that approached us to tell how they love Home Assistant, how it has replaced their previous solution, how they enjoyed contributing to Home Assistant and how helpful our community is. It makes me proud of Home Assistant and especially our community.

It’s time for release 0.21 and it contains massive core improvements: replacement of our home grown HTTP stack with a standardized WSGI stack. This will improve performance, speed, security and make future development of advanced HTTP features a breeze.

This work was driven by the amazing Josh Wright. His knowledge, high standards and drive for security has helped improve Home Assistant a lot ever since he started helping out. Hip hip hurray for Josh!

Alright, time for the changes:

• New HTTP stack based on WSGI (@JshWright, @balloob)
• Frontend: lots of performance improvements (@balloob)
• Initial support for EnOcean lights, sensors and switches added (@rubund)
• Light: Osram Lightify is now supported (@olimpiurob)
• Light: Insteon Hub now supports brightness (@wkonkel)
• Add support for adding HA as Windows 10 tile (@fabaff)
• Condition: time condition can now cross midnight (@jaharkes)
• Template based sensors should now throw less warnings (@Bart274)
• New LIRC component allows receiving IR commands (@partofthething)
• The Feedreader component will now persist data to disk to prevent duplicate events (@shaftoe)
• Sun: azimuth attribute added (@fabaff)
• New Flux like switch platform to change light intensity in the evening (@nkgilley)
• We no longer crash if you live in a part of the world where the sun never sets (@balloob)
• Rollershutter: RFXTRX now supported (@jacobtomlinson)
• Switch: Template switches can now execute scripts (@kellerza)
• Z-Wave: automatically heal the network at midnight (@infamy)
• Sensor: DTE Energy Bridge now supported (@kylehendricks)
• Media Player: Kodi now supports different turn off commands (@armills)

Breaking Changes

• Our work in the WSGI stack is not fully done yet. We still have a minor issues where retrieving the error log in the about screen can raise an encoding error
• The API used to incorrectly accept a JSON body with form-url-encoded headers. Our cURL examples on the website used to be wrong and have been updated.
• Make sure your configuration.yaml file contains frontend: to serve the frontend

Hotfixes 0.21.1 and 0.21.2

We released two hotfixes to address some issues that couldn’t wait till the next release.

0.21.1 - June 12

• Add eventlet to base requirements to resolve some installation issues (@balloob)
• GTFS will filter out routes in the wrong direction (@imrehg)
• Recover from rare error condition from LIRC (@partofthething)
• Z-Wave autoheal will no longer raise exception (@balloob)
• Alexa will now execute the script before making reply (@balloob)
• Fix MJPEG camera streaming (@stjohnjohnson)
• Fix frontend in older browsers (@balloob)
• Fix history in more info dialog being cut off (@balloob)

0.21.2 - June 15

• Fix input_select calling the set_option service again when changed (@persandstrom)
• Fix more info dialog not being able to open on Safari (@balloob)
• Add support for OPTIONS HTTP command to get CORS working (@JshWright)

Our community is amazingly helpful and creative. If you haven’t been there yet, make sure to stop by our chat room and come hang out with us. In this blog post I want to highlight a few recent awesome projects and videos from the community.

SceneGen - cli for making scenes

SceneGen is a new command line utility developed by Andrew Cockburn that helps with creating scene configurations for Home Assistant. To use it, you put your house in the preferred state, run SceneGen and it will print the scene configuration for your current states.

Videos

Nick Touran has been working on integrating IR remotes with Home Assistant. He made it into a component which should be available in the next release which should arrive in a couple of days. In the meanwhile, he wrote up a blog post and has put out a video showing the new integration, very cool!

Ben from BRUH Automation has put out another great video how to get started tracking your location in Home Assistant using MQTT and OwnTracks.

Muhammed Kilic has created a video how to make your Home Assistant instance accessible from the internet using the free dynamic DNS service DuckDNS.