Many home automation solutions provide a central dashboard that serves as a nerve center for the Internet of Things (IoT) environment. From the dashboard, you can monitor and manage light levels, thermostat temperatures, and all the other settings the system controls. Some users don’t realize that the dashboard can also provide information from beyond your home network. In other words, you can use the tools of the home automation environment to fetch information from the Internet and display it in a handy at-a-glance view along with your IoT settings.
For instance, you might have personal or hobby topics that you check periodically. These personal data points might be the wave height at a local surf spot, the insect levels at a favorite camping area, or the best time of go fishing. You probably know exactly where to find this information, however, it still requires several steps to start up a web browser, click on a bookmark, and scan the page for the desired data – which might just consist of a single number or a couple of words.
In this article, I will look at how to scrape these hobby data points from web pages with just a single line of Bash code and display the results in the dashboards of two home IoT packages, Home Assistant [1] and Node-RED [2].
Getting Started
Several different automation solutions offer web-scraping tools. For example, Python has the Beautiful Soup [3] library, Home Assistant has the Scrape [4] sensor, and Node-RED has the scrape-it [5] flow. These web-scraping tools are all very usable, but unfortunately they require a detailed knowledge of the HTML/ Document Object Model for the requested page.
Denne historien er fra #271/June 2023: Smart Home-utgaven av Linux Magazine.
Start din 7-dagers gratis prøveperiode på Magzter GOLD for å få tilgang til tusenvis av utvalgte premiumhistorier og 9000+ magasiner og aviser.
Allerede abonnent ? Logg på
Denne historien er fra #271/June 2023: Smart Home-utgaven av Linux Magazine.
Start din 7-dagers gratis prøveperiode på Magzter GOLD for å få tilgang til tusenvis av utvalgte premiumhistorier og 9000+ magasiner og aviser.
Allerede abonnent? Logg på
Tracking your finances with plain text accounting Plain Numbers
If you're tired of tinkering with spreadsheets, using hledger and plain text accounting offers a simpler method for managing your finances without vendor lock-in
Dependency resolution with apt-get and apt Evolutionary Tale
Over the past 30 years, the apt family has played an important role in dependency resolution for Debian distros.
Cryptomining with Litecoin Traveling Lite
Although not as popular as headliners like Bitcoin and Ethereum, Litecoin is one of the oldest crytocurrencies, and it offers some useful features, such as dual-mining with Dogecoin.
Software Update SnoopGod
SnoopGod delivers an Ubuntu-based pentesting distribution with an emphasis on security education.
Kernel Trouble
This deep look at how intruders attack an out-of-date kernel should be enough to convince you of the need to stay vigilant.
Using Wake-on-LAN for a NAS backup Power Saver
Put your backup server to sleep when you don't need it and then wake it on demand using the Wake-on-LAN feature built into network adapters.
Time Travel
Mike Schilli uses a Go program to check whether a strategy for trading stocks is making gains or losses on the basis of historical price data.
URL filtering with Pi-hole Into the Funnel
Supporting browser plug-ins, network-based DNS blockers like Pi-hole help protect you against online tracking and unwanted content.
Artificial intelligence on the Raspberry Pi Learning Experience
You don't need a powerful computer system to use Al. We show what it takes to benefit from Al on the Raspberry Pi and what tasks the small computer can handle.
MakerSpace Manage your greenhouse with a Raspberry Pi Pico W Sheltered Growth
You can safely assign some greenhouse tasks to a Raspberry Pi Pico W, such as controlling ventilation, automating a heater, and opening and closing windows.
