Month: August 2022

MQTT Data to Azure Log Analytics

Setting up solar on our farm in North Texas has been a treasure trove of data to analyze – everything from Inverter load, to PV watts in, to state-of-charge for the batteries – It’s a massive amount of useful information.

So naturally I want that data in Azure Log Analytics. It’s perfect platform to handle this amount of data. Small messages, lots of entries. Just ready for reporting and analytics.

To do this, we are going to use IOT Hub as a go-between. Here are the high-level steps:

To get MQTT data into Azure Log Analytics via IoT Hub, you will need to perform the following steps:

  1. Set up an IoT hub in your Azure account.
  2. Set up a device in your IoT hub.
  3. Configure the device to send data to your IoT hub using the MQTT protocol.
  4. Set up a log analytics workspace in your Azure account.
  5. Connect your IoT hub to your log analytics workspace.
  6. Start sending MQTT data from your device to your IoT hub.

And now some details:

  1. Set up an IoT hub in your Azure account:
  • In the Azure portal, click on “Create a resource” in the top left corner, then search for “IoT Hub” and select it.
  • Follow the prompts to create a new IoT hub, including selecting the subscription, resource group, and region you want to use.
  • Make note of the IoT hub’s name and connection string, as you will need these later.
  1. Set up a device in your IoT hub:
  • In the Azure portal, go to the IoT hub blade and select “IoT Devices” from the left menu.
  • Click the “Add” button to create a new device.
  • Follow the prompts to set up the device, including giving it a unique device ID and generating a device key or certificate.
  • Make note of the device ID and key/certificate, as you will need these later.
  1. Configure the device to send data to your IoT hub using MQTT:
  • The specific steps for this will depend on the device you are using, but generally you will need to specify the MQTT endpoint and authentication information for your IoT hub.
  • The MQTT endpoint will be in the format “YOUR-IOT-HUB-NAME.azure-devices.net”, and you will need to use the device ID and key/certificate you obtained in step 2 to authenticate.
  • Specify the MQTT topic you want to publish data to.
  1. Set up a log analytics workspace:
  • In the Azure portal, click on “Create a resource” in the top left corner, then search for “Log Analytics” and select it.
  • Follow the prompts to create a new log analytics workspace, including selecting the subscription, resource group, and region you want to use.
  • Make note of the workspace ID and primary key, as you will need these later.
  1. Connect your IoT hub to your log analytics workspace:
    • In the Azure portal, go to the IoT hub blade and select “Diagnostic settings” from the left menu.
    • Click the “Add diagnostic setting” button to create a new setting.
    • Select the log analytics workspace you created in step 4 as the destination for the data.
    • Select the data types you want to collect
    1. Start sending MQTT data from your device to your IoT hub:
    • Using the device ID, key/certificate, MQTT endpoint, and topic you obtained in steps 2 and 3, publish data to your IoT hub using the MQTT protocol.
    • The data should be automatically forwarded to your log analytics workspace and be available for analysis.