Manage IoT Sensor Network with ISTSOS (Open Source)

IstSOS is Free and Open Source Sensor Observation Service Data Management System (OGC SOS server implementation) written in Python. istSOS allows for managing and dispatch observations from monitoring sensors according to the Sensor Observation Service standard. It provides a Graphical user Interface that allows for easing the daily operations and a RESTFull Web api for automatizing administration procedures. istSOS is released under the GPL License, and runs on all major platforms (Windows, Linux, Mac OS X), even though it has been used in production in linux environment only.

Main Features

  • Offer your data according to the Sensor Observation Service standard from Open Geospatial Consortium.
  • Administer your sensors and your data with a comfortable interface.
  • Use a complete API for accessing functionalities to makes it easy for new clients to use istSOS application.
  • Get notified trough mail, twitter or other social when your sensor data met specific conditions.

How to Install (On Ubuntu 18.04)

Installation on Linux with the Debian package

The easiest way to install istSOS on a Debian distribution is to use the istSOS deb packages.

Download the debian file from the repository

Please go to https://sourceforge.net/projects/istsos/files to get the latest release.

Install the debian file

Open a terminal and move to the folder containing the downloaded deb file.

sudo dpkg -i python3-istsos_2.4.0-RC4.deb;sudo apt-get -f -y install

This command will install all the required dependencies, with the exception of PostgreSQL and PostGIS. In fact it could reside on other servers.

If everything has gone well, you should see the administration page at this address: http://localhost/istsos/admin/

NodeMCU ESP8266 Over The Air (OTA) Programming In Arduino IDE

A fantastic feature of any WiFi-enabled microcontroller like ESP8266 NodeMCU is the ability to update its firmware wirelessly. This is known as Over-The-Air (OTA) programming.

The OTA programming allows updating/uploading a new program to ESP8266 using Wi-Fi instead of requiring the user to connect the ESP8266 to a computer via USB to perform the update.

OTA functionality is extremely useful in case of no physical access to the ESP module. It helps reduce the amount of time spent for updating each ESP module at the time of maintenance.

One important feature of OTA is that one central location can send an update to multiple ESPs sharing same network.

The only disadvantage is that you have to add an extra code for OTA with every sketch you upload, so that you’re able to use OTA in the next update.

The factory image in ESP8266 doesn’t have an OTA Upgrade capability. So, you need to load the OTA firmware on the ESP8266 through serial interface first.

It’s a mandatory step to initially update the firmware, so that you’re able to do the next updates/uploads over-the-air.

The ESP8266 add-on for the Arduino IDE comes with a OTA library & BasicOTA example. You can access it through File > Examples > ArduinoOTA > BasicOTA.

The following code should load. But, before you head for uploading the sketch, you need to make some changes to make it work for you. You need to modify the following two variables with your network credentials, so that ESP8266 can establish a connection with existing network.

const char* ssid = "..........";
const char* password = "..........";

Once you are done, go ahead and upload the sketch.

 

#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>
#include <WiFiUdp.h>
#include <ArduinoOTA.h>

const char* ssid = "..........";
const char* password = "..........";

void setup() {
  Serial.begin(115200);
  Serial.println("Booting NodeMCU");
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.println("Sorry, Connection Failed! Rebooting NodeMCU...");
    delay(5000);
    ESP.restart();
  }
  ArduinoOTA.onStart([]() {
    String type;
    if (ArduinoOTA.getCommand() == U_FLASH)
      type = "sketch";
    else 
      type = "filesystem";

    Serial.println("Start updating " + type);
  });
  ArduinoOTA.onEnd([]() {
    Serial.println("\nEnd");
  });
  ArduinoOTA.onProgress([](unsigned int progress, unsigned int total) {
    Serial.printf("Progress: %u%%\r", (progress / (total / 100)));
  });
  ArduinoOTA.onError([](ota_error_t error) {
    Serial.printf("Error[%u]: ", error);
    if (error == OTA_AUTH_ERROR) Serial.println("Auth Failed");
    else if (error == OTA_BEGIN_ERROR) Serial.println("Begin Failed");
    else if (error == OTA_CONNECT_ERROR) Serial.println("Connecting Failed");
    else if (error == OTA_RECEIVE_ERROR) Serial.println("Receiving Failed");
    else if (error == OTA_END_ERROR) Serial.println("End Failed");
  });
  ArduinoOTA.begin();
  Serial.println("Ready");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  ArduinoOTA.handle();
}

Now, open the Serial Monitor at a baud rate of 115200. And press the RST button on ESP8266. If everything is OK, it will output the dynamic IP address obtained from your router. Note it down.

Upload New Sketch Over-The-Air

Now, let’s upload a new sketch over-the-air.

Remember! you need to add the code for OTA in every sketch you upload. Otherwise, you’ll loose OTA capability and will not be able to do next uploads over-the-air. So, it’s recommended to modify the above code to include your new code.

As an example we will include a simple Blink sketch in the Basic OTA code. Remember to modify the SSID and password variables with your network credentials.

#include <ESP8266WiFi.h>
#include <ESP8266mDNS.h>
#include <WiFiUdp.h>
#include <ArduinoOTA.h>

const char* ssid = "..........";
const char* password = "..........";

//variabls for blinking an LED with Millis
const int led = D0; // ESP8266 Pin to which onboard LED is connected
unsigned long previousMillis = 0;  // will store last time LED was updated
const long interval = 1000;  // interval at which to blink (milliseconds)
int ledState = LOW;  // ledState used to set the LED
void setup() {
pinMode(led, OUTPUT);
    
  Serial.begin(115200);
  Serial.println("Booting NodeMCU");
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.waitForConnectResult() != WL_CONNECTED) {
    Serial.println("Connection Failed! Rebooting NodeMCU...");
    delay(5000);
    ESP.restart();
  }
  ArduinoOTA.onStart([]() {
    String type;
    if (ArduinoOTA.getCommand() == U_FLASH)
      type = "sketch";
    else
      type = "filesystem";

    Serial.println("Start updating " + type);
  });
  ArduinoOTA.onEnd([]() {
    Serial.println("\nEnd");
  });
  ArduinoOTA.onProgress([](unsigned int progress, unsigned int total) {
    Serial.printf("Progress: %u%%\r", (progress / (total / 100)));
  });
  ArduinoOTA.onError([](ota_error_t error) {
    Serial.printf("Error[%u]: ", error);
    if (error == OTA_AUTH_ERROR) Serial.println("Auth Failed");
    else if (error == OTA_BEGIN_ERROR) Serial.println("Begin Failed");
    else if (error == OTA_CONNECT_ERROR) Serial.println("Connecting Failed");
    else if (error == OTA_RECEIVE_ERROR) Serial.println("Receiving Failed");
    else if (error == OTA_END_ERROR) Serial.println("End Failed");
  });
  ArduinoOTA.begin();
  Serial.println("Ready");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
}

void loop() {
  ArduinoOTA.handle();

  unsigned long currentMillis = millis();
  if (currentMillis - previousMillis >= interval) {

  previousMillis = currentMillis;
  ledState = not(ledState);
  digitalWrite(led,  ledState);
  }
}

Once you copy above sketch to your Arduino IDE, go to Tools > Port option and you should see something like this: esp8266-xxxxxx at your_esp_ip_address If you can’t find it, you may need to restart your IDE.

Select the port and click Upload button. Within a few seconds, the new sketch will be uploaded. And you should see the on-board LED blinking.

Smart Switch using NodeMCU – Works with Google Home Assistant

Google assistant is AI (Artificial Intelligence) based voice command service. Using voice, we can interact with google assistant and it can search on the internet, schedule events, set alarms, control appliances, etc. This service is available on smartphones and Google Home devices.

We can control smart home devices including lights, switches, fans, thermostats and many more devices using our Google Assistant.

A lot of commercial devices already exist in the market for such application but,. Here i have explained how to build your own smart switch using NodeMCU (ESP8266) development board and few other electrical components.

Components used;

  • NodeMCU (ESP8266) Development Board.
  • Solid State Relay Module.
  • 6V Power Supply Module.
  • Connection Wires.

Above figure explained very simply about the circuit diagram. Be aware when you connecting this to the main power. If you don’t have enough knowledge about connecting this to the main power system, please get some help from a experienced  technician.

Before you connect using above explained diagram, you must programme the NodeMCU development board. Otherwise you cannot control it using Google Smart Home Assistant with voice commands.

In above diagram, i used D0 – Digital pin to control the input signal connected to the Solid State Relay module. When we want to switch on the light, we need to set it to high voltage state (5V), and when we want to switch of the light, we need to set it to low voltage state (0V).

You can find the source code which needs to be uploaded to the NodeMCU development borad using Arduino IDE here. https://github.com/lahirutm/WebserviceLK-IoT/blob/master/arduino_examples/state_save_light_example.ino

Once you programmed your NodeMCU, you must link it with Google Home Assistant to control using voice commands. Please follow the steps explained here.  https://iot.webservice.lk/how_to

Enjoy using your own smart home devices…

Toroidal Transformers

Transformers are made from a pair of solenoids wrapped around a metal core that is usually a ferrite. Toroidal transformers are two coils wrapped around a metal, such as a ferrite or silicon steel, that is doughnut shaped. The coils are either wrapped in different areas or placed one over the other. They are preferred for RF or radio frequency transformers, where they are used to increase or decrease voltages from power sources, and to isolate different parts in a circuit. RF transformers are also used for impedance matching, which means they help connect input and output parts of different circuits.

Here are five other reasons why toroidal transformers are used for many industrial applications:

  1. Low noise and low stray field – The field generated by magnetization, also known as the stray field, is lower in a toroidal transformer, due to the uniform core windings. Less magnetic interference in toroids results in higher performance.

  2. Easy to mount – Using just one screw, toroidal transformers can be easily mounted in a short time. This convenience helps limit maintenance and downtime.

  3. Low operating temperature – Toroidal transformers operate at lower temperatures than transformers with similar specifications.

  4. Light weight core – The core of a toroidal transformer weighs less than typical transformers due to it being comprised of less raw materials.

  5. More economical – Since toroidal transformer cores are manufactured from fewer materials, they weigh less than conventional transformers and use less energy, providing better cost savings and higher return on investment.

The toroidal transformers efficiency makes them useful for a wide array of machines such as audio/visual equipment, security systems, telecommunication systems, industrial control equipment and power distribution equipment. Cost efficiency is an important key to the widespread use of toroidal transformers, as well as the fact that they can be customized for any diameter and height.

Difference Between MPPT and PWM Charge Controller

 

MPPT Solar Charge Controller

PWM Solar Charge Controller

1. What they do

The PWM controller is in essence a switch that connects a solar array to a battery. The result is that the voltage of the array will be pulled down to near that of the battery.

The MPPT controller is more sophisticated (and more expensive): it will adjust its input voltage to harvest the maximum power from the solar array and then transform this power to supply the varying voltage requirement, of the battery plus load. Thus, it essentially decouples the array and battery voltages so that there can be, for example, a 12 volt battery on one side of the MPPT charge controller and a large number of cells wired in series to produce 36 volts on the other.

As array size increases, cable length will increase. The option to wire more panels in series and thereby decrease the cable cross sectional area with a resultant drop in cost, is a compelling reason to install an MPPT controller as soon as the array power exceeds a few hundred Watts (12 V battery), or several 100s of Watts (24 V or 48 V battery).

PWM

The PWM charge controller is a good low cost solution for small systems only, when solar cell temperature is moderate to high (between 45°C and 75°C).

MPPT

To fully exploit the potential of the MPPT controller, the array voltage should be substantially higher than the battery voltage. The MPPT controller is the solution of choice for higher power systems (because of the lowest overall system cost due to smaller cable cross sectional areas). The MPPT controller will also harvest substantially more power when the solar cell temperature is low (below 45°C), or very high (above 75°C), or when irradiance is very low.

  • Charging Method
    The difference between the PWM controller and the MPPT solar controller is that they charge differently. The PWM is charged in a three-stage charging mode. MPPT is the maximum power tracking technology, and the charging efficiency can be increased to about 30%.
  • Voltage
    When the MPPT solar controller is used, the charging efficiency is more obvious when the high-voltage solar panel is used to charge the low-voltage battery. If the voltage of the solar energy and the battery are the same, for example, it is 12V, then the difference between the practical PWM and the MPPT controller is not very big.
  • Power
    If the power of the solar panel is small, it is more appropriate to select a PWM controller. Relatively speaking, if the power of the solar panel is large, MPPT is selected.Solar controllers, because the cost of MPPT solar controllers is high, if your solar panel power is small, it is a waste of MPPT controller.

Salt Water Batteries

Saltwater batteries are not similar to lithium-ion batteries or lead acid batteries for use in portable devices or any automobiles. This is because they can’t hold as much charge in the same size and weight, or putting it another way, they are less energy-dense. But they hold some powerful advantages in applications in which size and weight are less important.

The electrolyte for a saltwater battery is nothing more than — salt water — hence the device’s name comes from it. The anode can be carbon, and the cathode can be a material such as manganese oxide. (Or anode can be copper and cathode can be zinc or aluminum).

With no hazardous materials in their construction, unlike lithium-ion batteries, they are non-toxic and they cannot explode. There is no need for the complex and troublesome electronic circuitry that every lithium-ion battery needs to ensure that they charge and discharge only within safe parameters. In addition, unlike their lithium-based cousins, saltwater batteries can be more deeply discharged (drained of electrical energy) with no damage to the battery.

The biggest disadvantage of all forms of salt-water batteries is that, to store a given amount of electricity, they are bulkier and heavier than other commercially available batteries. As it turns out, there is one huge potential use for these devices in which that will be no issue at all — smoothing out power from large-scale renewable-energy-generation plants.

Get selected location’s coordinates using Google’s Javascript API

To get the coordinates from a selected location on the map (Latitude and Longitude) to input fields, we can use Google’s Javascript MAP API. Here i have used a search field to search locations by name. Therefor i had to use Places API too.

How it works

I have used simple Bootstrap4 html page to place map. According to the Google’s documentation guide we can use below Javascript codes to get selected location’s coordinates in to two input fields at the bottom.

 

<script src="https://maps.googleapis.com/maps/api/js?key=AIzaSyAcr__t5RD7E40Yz6rgEVHQOAI4kcsMrIA&libraries=places&callback=initialize" async defer></script>
<script>
function initialize() {
	var myLatlng = new google.maps.LatLng(7.873053999999999,80.77179699999999);
	var myOptions = {
		zoom: 8,
		center: myLatlng,
		mapTypeId: google.maps.MapTypeId.ROADMAP
	}

	var map = new google.maps.Map(document.getElementById('map-canvas'), myOptions);

	// Create the search box and link it to the UI element.
	var input = document.getElementById('pac-input');
	var searchBox = new google.maps.places.SearchBox(input);
	map.controls[google.maps.ControlPosition.TOP_LEFT].push(input);

	var marker = new google.maps.Marker({
		draggable:true,
		position: myLatlng,
		map: map
	});

	// Bias the SearchBox results towards current map's viewport.
	map.addListener('bounds_changed', function() {
		searchBox.setBounds(map.getBounds());
	});

	google.maps.event.addListener(marker, 'dragend', function (event) {
		var poiLat = this.getPosition().lat();
		var poiLon = this.getPosition().lng();
		$('#latitude').val(poiLat);
		$('#longitude').val(poiLon);
	});

	var markers = [];
	// Listen for the event fired when the user selects a prediction and retrieve
	// more details for that place.
	searchBox.addListener('places_changed', function() {
		var places = searchBox.getPlaces();

		if (places.length == 0) {
			return;
		}

		// Clear out the old markers.
		markers.forEach(function(marker) {
			marker.setMap(null);
		});
		markers = [];

		// For each place, get the icon, name and location.
		var bounds = new google.maps.LatLngBounds();
		places.forEach(function(place) {
			if (!place.geometry) {
				console.log("Returned place contains no geometry");
				return;
			}

			var icon = {
				url: place.icon,
				size: new google.maps.Size(71, 71),
				origin: new google.maps.Point(0, 0),
				anchor: new google.maps.Point(17, 34),
				scaledSize: new google.maps.Size(25, 25)
			};

			// Create a marker for each place.
			marker.setPosition( place.geometry.location );
			map.panTo( place.geometry.location );
			$('#latitude').val(place.geometry.location.lat());
			$('#longitude').val(place.geometry.location.lng());

			if (place.geometry.viewport) {
			// Only geocodes have viewport.
				bounds.union(place.geometry.viewport);
			} else {
				bounds.extend(place.geometry.location);
			}
		});

		map.fitBounds(bounds);
	});
}

</script>

Here is the final output.

You can download the source code from my Github: https://github.com/lahirutm/Get-coordinates-using-Google-s-Javascript-API

Convert video files using FFMPEG in Linux

Convert MKV to MP4

ffmpeg -i video_file.mkv output_file.mp4

If you only want to convert MKV to MP4 then you will save quality and a lot of time by just changing the containers. Both of these are just wrappers over the same content so the CPU only needs to do a little work. Don’t re encode as you will definitely lose quality.

It’s very straight forward using ffmpeg:

ffmpeg -i LostInTranslation.mkv -codec copy LostInTranslation.mp4

Here, you are copying the video codec and audio codec so nothing is being encoded.

To convert all MKV files in the current directory, run a simple loop in terminal:

for i in *.mkv; do
    ffmpeg -i "$i" -codec copy "${i%.*}.mp4"
done

Note: If you get an error like this:

opus in MP4 support is experimental, add '-strict -2' if you want to use it.
Could not write header for output file #0 (incorrect codec parameters ?): Experimental feature

use

for i in *.mkv; do
ffmpeg
-i "$i" -codec copy -strict -2 "${i%.*}.mp4" done

 

Convert MP4 to AVI

In order to just copy the video and audio bitstream, thus without quality loss:

ffmpeg -i filename.mkv -c:v copy -c:a copy output.avi

If you want FFmpeg to convert video and audio automatically:

ffmpeg -i filename.mkv output.avi

Blinking LED With Arduino Nano

Things you need: Arduino Nano, 3V LED, 100 Ohm Resistor

To power up the Arduino Nano board, you can use USB cable or an external power supply (7V ~ 12V DC) by connecting positive pin to VIN and and negative to the ground. Please see my previous post about powering up Arduino Nano here: https://blogs.webservice.lk/2020/03/17/powering-up-arduino-nano/

To connect LED, use LED’s positive end to the one end of resistor and other end to Arduino Nano’s digital pin 13 & negative end to the  one of Arduino Nano’s ground pin. Please see the picture above.

Now circuit making part is done, Let’s do the programing.

int ledPin =  13;// the number of the LED pin

void setup() {
  // set the digital pin as output:
  pinMode(ledPin, OUTPUT);
}

void loop() {
  digitalWrite(ledPin, HIGH);   // turn the LED on (HIGH is the voltage level)
  delay(1000);                       // wait for a second
  digitalWrite(ledPin, LOW);    // turn the LED off by making the voltage LOW
  delay(1000);
}

Upload above sketch to the Arduino Nano and see the result. Led will blink with the 1 second delay.

Powering Up Arduino Nano

The Arduino Nano is a small, complete, and breadboard-friendly board based on the ATmega328P (Arduino Nano 3.x). It has more or less the same functionality of the Arduino Duemilanove, but in a different package. It lacks only a DC power jack, and works with a Mini-B USB cable instead of a standard one.

The Arduino Nano can be powered via the Mini-B USB Jack (5V) connection. 

 

The Arduino Nano accepts the 7V to 12V input power from the Vin pin (pin30).

 

If you want to supply regulated power, then a 5V regulated adapter needs to feed the +5V pin (pin27) instead.

 

The power source is automatically selected to the highest voltage source if you input multiple power inputs.