Merge branch 'development03' into eth

update to latest release 0.7.26

Getting_Started.md

@@ -293,10 +293,9 @@ When everything is wired up and the firmware is flashed, it is time to connect t
 | `ESP Async WebServer` | 1.2.3   | LGPL-3.0 |
 | `ESPAsyncTCP`         | 1.2.2   | LGPL-3.0 |
 | `Time`                | 1.6.1   | LGPL-2.1 |
-| `RF24`                | 1.4.5   | GPL-2.0  |
+| `RF24`                | 1.4.7   | GPL-2.0  |
-| `PubSubClient`        | 2.8     | MIT      |
+| `espMqttClient`       | 1.4.4   | MIT      |
-| `ArduinoJson`         | 6.19.4  | MIT      |
+| `ArduinoJson`         | 6.21.3  | MIT      |
-| `ESP Async WebServer` | 4.3.0   | ?        |
 
 ## ToDo
 

README.md

@@ -49,6 +49,4 @@ If you run into any issues, please feel free to use the issue tracker here on Gi
 
 ### Related Projects
 - [OpenDTU](https://github.com/tbnobody/OpenDTU)
-  <- Our sister project ✨ for Hoymiles HM-300, HM-600, HM-1200 (for ESP32 only!)
+  <- Our sister project ✨ for Hoymiles HM- and HMS-/HMT-series (for ESP32 only!)
-- [DTU Simulator](https://github.com/Ziyatoe/DTUsimMI1x00-Hoymiles) 
-  <- Go here ✨ for Hoymiles MI-300, MI-600, MI-1200 Software (single inverter only)

src/CHANGES.md

@@ -1,136 +1,3 @@
-# Development Changes
+Changelog v0.7.26
 
-## 0.7.23 - 2023-08-04
+* fix MqTT `last_success`
-* merge PR #1056, visualization html
-* update MqTT library to 1.4.4
-* update RF24 library to 1.4.7
-* update ArduinoJson library to 6.21.3
-* set minimum invervall for `/live` to 5 seconds
-
-## 0.7.22 - 2023-08-04
-* attempt to fix homeassistant auto discovery #1066
-
-## 0.7.21 - 2023-07-30
-* fix MqTT YieldDay Total goes to 0 serveral times #1016
-
-## 0.7.20 - 2023-07-28
-* merge PR #1048 version and hash in API, fixes #1045
-* fix: no yield day update if yield day reads `0` after inverter reboot (mostly on evening) #848
-* try to fix Wifi override #1047
-* added information after NTP sync to WebUI #1040
-
-## 0.7.19 - 2023-07-27
-* next attempt to fix yield day for multiple inverters #1016
-* reduced threshold for inverter state machine from 60min to 15min to go from state `WAS_ON` to `OFF`
-
-## 0.7.18 - 2023-07-26
-* next attempt to fix yield day for multiple inverters #1016
-
-## 0.7.17 - 2023-07-25
-* next attempt to fix yield day for multiple inverters #1016
-* added two more states for the inverter status (also docu)
-
-## 0.7.16 - 2023-07-24
-* next attempt to fix yield day for multiple inverters #1016
-* fix export settings date #1040
-* fix time on WebUI (timezone was not observed) #913 #1016
-
-## 0.7.15 - 2023-07-23
-* add NTP sync interval #1019
-* adjusted range of contrast / luminance setting #1041
-* use only ISO time format in Web-UI #913
-
-## 0.7.14 - 2023-07-23
-* fix Contrast for Nokia Display #1041
-* attempt to fix #1016 by improving inverter status
-* added option to adjust effiency for yield (day/total) #1028
-
-## 0.7.13 - 2023-07-19
-* merged display PR #1027
-* add date, time and version to export json #1024
-
-## 0.7.12 - 2023-07-09
-* added inverter status - state-machine #1016
-
-## 0.7.11 - 2023-07-09
-* fix MqTT endless loop #1013
-
-## 0.7.10 - 2023-07-08
-* fix MqTT endless loop #1013
-
-## 0.7.9 - 2023-07-08
-* added 'improve' functions to set wifi password directly with ESP web tools #1014
-* fixed MqTT publish while appling power limit #1013
-* slightly improved HMT live view (Voltage & Current)
-
-## 0.7.8 - 2023-07-05
-* fix `YieldDay`, `YieldTotal` and `P_AC` in `TotalValues` #929
-* fix some serial debug prints
-* merge PR #1005 which fixes issue #889
-* merge homeassistant PR #963
-* merge PR #890 which gives option for scheduled reboot at midnight (default off)
-
-## 0.7.7 - 2023-07-03
-* attempt to fix MqTT `YieldDay` in `TotalValues` #927
-* attempt to fix MqTT `YieldDay` and `YieldTotal` even if inverters are not completly available #929
-* fix wrong message 'NRF not connected' if it is disabled #1007
-
-## 0.7.6 - 2023-06-17
-* fix display of hidden SSID checkbox
-* changed yield correction data type to `double`, now decimal places are supported
-* corrected name of 0.91" display in settings
-* attempt to fix MqTT zero values only if setting is there #980, #957
-* made AP password configurable #951
-* added option to start without time-sync, eg. for AP-only-mode #951
-
-## 0.7.5 - 2023-06-16
-* fix yield day reset on midnight #957
-* improved tickers in `app.cpp`
-
-## 0.7.4 - 2023-06-15
-* fix MqTT `P_AC` send if inverters are available #987
-* fix assignments for HMS 1CH and 2CH devices
-* fixed uptime overflow #990
-
-## 0.7.3 - 2023-06-09
-* fix hidden SSID scan #983
-* improved NRF24 missing message on home screen #981
-* fix MqTT publishing only updated values #982
-
-## 0.7.2 - 2023-06-08
-* fix HMS-800 and HMS-1000 assignments #981
-* make nrf enabled all the time for ESP8266
-* fix menu item `active` highlight for 'API' and 'Doku'
-* fix MqTT totals issue #927, #980
-* reduce maximum number of inverters to 4 for ESP8266, increase to 16 for ESP32
-
-## 0.7.1 - 2023-06-05
-* enabled power limit control for HMS / HMT devices
-* changed NRF24 lib version back to 1.4.5 because of compile problems for EPS8266
-
-## 0.7.0 - 2023-06-04
-* HMS / HMT support for ESP32 devices
-
-## 0.6.15 - 2023-05-25
-* improved Prometheus Endpoint PR #958
-* fix turn off ePaper only if setting was set #956
-* improved reset values and update MqTT #957
-
-## 0.6.14 - 2023-05-21
-* merge PR #902 Mono-Display
-
-## 0.6.13 - 2023-05-16
-* merge PR #934 (fix JSON API) and #944 (update manual)
-
-## 0.6.12 - 2023-04-28
-* improved MqTT
-* fix menu active item
-
-## 0.6.11 - 2023-04-27
-* added MqTT class for publishing all values in Arduino `loop`
-
-## 0.6.10 - HMS
-* Version available in `HMS` branch
-
-## 0.6.9
-* last Relaese

src/app.h

@@ -24,7 +24,7 @@
 #include "utils/crc.h"
 #include "utils/dbg.h"
 #include "utils/scheduler.h"
-#include "utils/improv.h"
+//#include "utils/improv.h"
 #include "web/RestApi.h"
 #include "web/web.h"
 #if defined(ETHERNET)

src/config/settings.h

@@ -128,6 +128,7 @@ typedef struct {
 typedef struct {
     char broker[MQTT_ADDR_LEN];
     uint16_t port;
+    char clientId[MQTT_CLIENTID_LEN];
     char user[MQTT_USER_LEN];
     char pwd[MQTT_PWD_LEN];
     char topic[MQTT_TOPIC_LEN];

src/defines.h

@@ -13,7 +13,7 @@
 //-------------------------------------
 #define VERSION_MAJOR       0
 #define VERSION_MINOR       7
-#define VERSION_PATCH       23
+#define VERSION_PATCH       26
 
 //-------------------------------------
 typedef struct {
@@ -85,6 +85,7 @@ enum {MQTT_STATUS_OFFLINE = 0, MQTT_STATUS_PARTIAL, MQTT_STATUS_ONLINE};
 #define NTP_ADDR_LEN            32 // DNS Name
 
 #define MQTT_ADDR_LEN           64 // DNS Name
+#define MQTT_CLIENTID_LEN       22 // number of chars is limited to 23 up to v3.1 of MQTT
 #define MQTT_USER_LEN           65 // there is another byte necessary for \0
 #define MQTT_PWD_LEN            65
 #define MQTT_TOPIC_LEN          65

src/publisher/pubMqtt.h

@@ -76,6 +76,10 @@ class PubMqtt {
 
             if((strlen(mCfgMqtt->user) > 0) && (strlen(mCfgMqtt->pwd) > 0))
                 mClient.setCredentials(mCfgMqtt->user, mCfgMqtt->pwd);
+            if(strlen(mCfgMqtt->clientId) > 0) {
+                snprintf(mClientId, 23, "%s-", mCfgMqtt->clientId);
+                mClient.setClientId(mCfgMqtt->clientId);
+            } else{
                 snprintf(mClientId, 24, "%s-", mDevName);
                 uint8_t pos = strlen(mClientId);
                 mClientId[pos++] = WiFi.macAddress().substring( 9, 10).c_str()[0];
@@ -87,6 +91,7 @@ class PubMqtt {
                 mClientId[pos++] = '\0';
 
                 mClient.setClientId(mClientId);
+            }
             mClient.setServer(mCfgMqtt->broker, mCfgMqtt->port);
             mClient.setWill(mLwtTopic, QOS_0, true, mqttStr[MQTT_STR_LWT_NOT_CONN]);
             mClient.onConnect(std::bind(&PubMqtt::onConnect, this, std::placeholders::_1));
@@ -466,7 +471,6 @@ class PubMqtt {
             bool anyAvail = false;  // shows if at least one enabled inverter is available
             bool changed = false;
             Inverter<> *iv;
-            record_t<> *rec;
 
             for (uint8_t id = 0; id < mSys->getNumInverters(); id++) {
                 iv = mSys->getInverterByPos(id);
@@ -475,11 +479,9 @@ class PubMqtt {
                 if (!iv->config->enabled)
                     continue; // skip to next inverter
 
-                rec = iv->getRecordStruct(RealTimeRunData_Debug);
-
                 // inverter status
                 iv->isProducing(); // recalculate status
-                if (iv->isAvailable())
+                if (InverterStatus::OFF < iv->status)
                     anyAvail = true;
                 else // inverter is enabled but not available
                     allAvail = false;
@@ -495,10 +497,6 @@ class PubMqtt {
                     snprintf(mSubTopic, 32 + MAX_NAME_LENGTH, "%s/available", iv->config->name);
                     snprintf(mVal, 40, "%d", (uint8_t)iv->status);
                     publish(mSubTopic, mVal, true);
-
-                    snprintf(mSubTopic, 32 + MAX_NAME_LENGTH, "%s/last_success", iv->config->name);
-                    snprintf(mVal, 40, "%d", iv->getLastTs(rec));
-                    publish(mSubTopic, mVal, true);
                 }
             }
 

src/publisher/pubMqttIvData.h

@@ -101,6 +101,11 @@ class PubMqttIvData {
 
             mPos = 0;
             if(found) {
+                record_t<> *rec = mIv->getRecordStruct(mCmd);
+                snprintf(mSubTopic, 32 + MAX_NAME_LENGTH, "%s/last_success", mIv->config->name);
+                snprintf(mVal, 40, "%d", mIv->getLastTs(rec));
+                mPublish(mSubTopic, mVal, true);
+
                 mIv->isProducing(); // recalculate status
                 mState = SEND_DATA;
             } else if(mSendTotals && mTotalFound)

src/web/RestApi.h

@@ -393,6 +393,7 @@ class RestApi {
 
         void getMqtt(JsonObject obj) {
             obj[F("broker")]     = String(mConfig->mqtt.broker);
+            obj[F("clientId")]   = String(mConfig->mqtt.clientId);
             obj[F("port")]       = String(mConfig->mqtt.port);
             obj[F("user")]       = String(mConfig->mqtt.user);
             obj[F("pwd")]        = (strlen(mConfig->mqtt.pwd) > 0) ? F("{PWD}") : String("");

src/web/html/api.js

@@ -30,6 +30,10 @@ iconSuccess = [
     "M10.97 4.97a.235.235 0 0 0-.02.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-1.071-1.05z"
 ];
 
+iconSuccessFull = [
+    "M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"
+];
+
  /**
  * GENERIC FUNCTIONS
  */

src/web/html/index.html

@@ -119,10 +119,12 @@
             function parseIv(obj) {
                 var p = div(["none"]);
                 for(var i of obj) {
-                    var icon = iconWarn;
+                    var icon = iconSuccess;
-                    var cl = "icon-warn";
+                    var cl = "icon-success";
-                    avail = "";
+                    var avail = "";
                     if(false == i["enabled"]) {
+                        icon = iconWarn;
+                        cl = "icon-warn";
                         avail = "disabled";
                     }
                     else if(false == i["is_avail"]) {
@@ -131,16 +133,14 @@
                         avail = "not yet available";
                     }
                     else if(0 == i["ts_last_success"]) {
-                        icon = iconSuccess;
                         avail = "available but no data was received until now";
                     }
                     else {
-                        icon = iconSuccess;
                         avail = "available and is ";
                         if(false == i["is_producing"])
                             avail += "not ";
                         else
-                            cl = "icon-success";
+                            icon = iconSuccessFull;
                         avail += "producing";
                     }
 

src/web/html/setup.html

@@ -249,6 +249,10 @@
                             <div class="col-12 col-sm-3 my-2">Port</div>
                             <div class="col-12 col-sm-9"><input type="number" name="mqttPort"/></div>
                         </div>
+                        <div class="row mb-3">
+                            <div class="col-12 col-sm-3 my-2">Client Id (optional)</div>
+                            <div class="col-12 col-sm-9"><input type="text" name="mqttClientId"/></div>
+                        </div>
                         <div class="row mb-3">
                             <div class="col-12 col-sm-3 my-2">Username (optional)</div>
                             <div class="col-12 col-sm-9"><input type="text" name="mqttUser"/></div>
@@ -696,7 +700,7 @@
             }
 
             function parseMqtt(obj) {
-                for(var i of [["Addr", "broker"], ["Port", "port"], ["User", "user"], ["Pwd", "pwd"], ["Topic", "topic"], ["Interval", "interval"]])
+                for(var i of [["Addr", "broker"], ["Port", "port"], ["ClientId", "clientId"], ["User", "user"], ["Pwd", "pwd"], ["Topic", "topic"], ["Interval", "interval"]])
                     document.getElementsByName("mqtt"+i[0])[0].value = obj[i[1]];
             }
 

src/web/web.h

@@ -594,6 +594,7 @@ class Web {
                 addr.toCharArray(mConfig->mqtt.broker, MQTT_ADDR_LEN);
             } else
                 mConfig->mqtt.broker[0] = '\0';
+            request->arg("mqttClientId").toCharArray(mConfig->mqtt.clientId, MQTT_CLIENTID_LEN);
             request->arg("mqttUser").toCharArray(mConfig->mqtt.user, MQTT_USER_LEN);
             if (request->arg("mqttPwd") != "{PWD}")
                 request->arg("mqttPwd").toCharArray(mConfig->mqtt.pwd, MQTT_PWD_LEN);