Multiscrape

So as you might have seen in my previous blog, I have some input numbers for my energy- and gas costs. But since they change like every month or so, I really wanted to automate that. And with that I give you Multiscrape!

Multiscrape is a HACS integration, which you can use to “scrape” data from websites. You can easily install this from HACS. I have done this already, and since installing integrations from HACS should be fairly straight forward, so I’m not gonna bother you with that.

So after installing the HACS integration, you need to add something to your config.yaml file. For me, for getting the prices I wanted I had to add the following:

multiscrape:
  - name: vattenfall scraper
    resource: "https://www.overstappen.nl/energie/leveranciers/vattenfall/"
    scan_interval: 3600
    log_response: false
    sensor:
      - unique_id: energieprijs_vattenfall
        name: Energieprijs Vattenfall
        select: "#tarieven > div.tableBlock__tableWrapper > table > tbody > tr > td"
        unit_of_measurement: "€/kWh"
        value_template: '{{ (((value.lstrip("€ ") | replace(",", ".")) | float) * 0.8) | round(6) }}'
      - unique_id: gasprijs_vattenfall
        name: Gasprijs Vattenfall
        select: "#tarieven > div.tableBlock__tableWrapper > table > tbody > tr:nth-child(4) > td"
        unit_of_measurement: "€/m3"
        value_template: '{{ ((value.lstrip("€ ") | replace(",", ".")) | float) }}'
      - unique_id: terugleververgoeding_vattenfall
        name: Terugleververgoeding Vattenfall
        select: "#tarieven > div.tableBlock__tableWrapper > table > tbody > tr:nth-child(5) > td"
        unit_of_measurement: "€/kWh"
        value_template: '{{ ((value.lstrip("€ ") | replace(",", ".")) | float) }}'

Explanaition

I will try to explain the things you see here.
The name kinda speaks for itself. The resource is the site from where you want to “scrape” the data.
Select is a CSS way of specifying the specific part of the site you want to obtain. The way this works is as follows:
First you go to the website from which you want the data. Then you open the inspector in your browser (I use Microsoft Edge). In there drill all the way down throught the tree of elements, until just the data you want is highlighted on screen. Right click that part, and select Copy -> Copy Selector. That way you get the line that you need to fill in at the “select” property in your config.

Unit of measurement kinda speaks for itself. And then the value template.
Since I’m scraping prices that have the format “€ 3,123” and I want to be able to do some calculations with them. I need to strip the “€ “. That is what the value.lstrip(“€ “) does. This strips from the left side. Then I need to replace the “,” with a “.” otherwise calculations won’t work on it. And by using “float” it tells Home Assistant to read is as a number instead of a string. The *0.8 part, is because on my energie costs, I get a 20% discount, because I’m a long term customer. And the “round(6)” part is to get a maximum of 6 decimals.

After setting this up, we reload the multiscrape entities (yes this is possible). And we end up with the following results:

Now all that is left is to encorporate this somewhere in our dashboard. I did it like this:

The “Energie kosten p…” and the “Gas kosten per d…” are imput numbers that I need for my calculations. I fill these in manually.

I wanted to get the prices directly from my supplier, however their website has a login form which cannot be scraped. Believe me I tried! So I went with a different website that has prices from multiple suppliers. Although they might nog be as up-to-date, they will do for now.

Well, thats about it for Multiscrape for me, at least for now. If you ever wonder what else is possible with it, just go visit the wiki for Multiscrape.

As always thank you for your time and interest in my ranting and I hope you enjoyed.