Imagine a system where the information is displayed the very moment it is generated. That is what we call the Real-time data.
Real-time data is information that has made available instantly after it has been collected. There might be a slight delay due to processing or transmission, but the goal is to get the data to the user as quickly as possible. Real-time data is useful in the situations where having the latest information is critical. It allows in quicker decision-making and can be used to monitor and react to changing situations. Example, Weather alert, Traffic alert, Stock market, Sensor data from IoT devices etc.
Cloud computing plays a significant role in enabling real-time data processing and visualization. Here is how we can use it:
Scalable Infrastructure: Cloud computing offers on-demand resources that means one can easily scale up and down the requirements whenever needed in rea-time. Thus, we there is an increase and decrease in the volume of real-time data stream the cloud platform can handle it easily.
Cost Effective: Cloud computing reduces the cost of the on-premises infrastructure. You only pay for the recourses that you use.
Real-time Monitoring: Cloud computing helps in real-time monitoring of the sensors and devices that helps in healthcare agriculture industries.
Understanding of AWS and its services:
Amazon Web Services (AWS) a comprehensive cloud computing platform offered by Amazon. It provides many computing resources like Computer Power, Storage, Databases, Networking, Analytics, Security, etc. AWS offers variety of services such as Management Tools (Monitoring and logging tools), Developer Tools (Source code management), Application Services (AWS Lambda, AWS API Gateway), Machine Learning, Mobile Services, Internet of Things (IoT), etc. Use of AWS has increased due to some of the best benefits such as cost-effective solutions. It is highly scalable, secure, and easy to use.
AWS plays a critical role in enabling real-time data processing by providing the infrastructure, tools, and services needed to handle the constant stream of information. AWS helps in building a dashboard where one can process and visualize the real-time data. It can be achieved using the following AWS services:
- AWS Lambda (For data processing): AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources.
- AWS RDS (For data storage):Amazon Relational Database Service (Amazon RDS) is an easy-to-manage relational database service optimized for total cost of ownership. It is simple to set up, operate, and scale with demand.
- AWS API Gateway (To fetch the data): Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale.
- AWS S3 (To deploy services): Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance.
While AWS provides number of services, we have mainly focused on the above-mentioned services.
Building Dashboards:
IoT dashboard gives the centralized view just like a command centre provides a single point to monitor and manage various aspects of an operation. It also provides real-time information about ongoing activities.
Follow the below mentioned steps to up an IoT device in AWS.
- Create an AWS Account
- Enable AWS IoT Core
- Create a Thing
- Create a Certificate
An IoT is a network of the physical objects that are known as Things.
AWS IoT Core is a managed cloud service that enables the connected devices to securely interact with the cloud applications, and other devices.
Certificates ensure only authorized devices (with the matching private key) can connect and exchange data with AWS IoT Core. This helps prevent unauthorized access and protects your device’s sensitive data. An X.509 certificate is needed to establish a secure connection between the IoT thing and AWS IoT Core. One can use the AWS IoT Console or AWS CLI to create the certificate (Thing certificate, private key and root CA certificate), endpoint address for specific Aws IoT Core instance, and we specify the specific MQTT topic on which the data is to be send and received.
Once the device is set up and powered up it publishes sensor data to a specific topic on the AWS IoT Core. To store the data in the database one can either use direct connection to the database to store without processing the data and can use the AWS Lambda to process the data and then store it in database.
Using AWS DynamoDB one can directly store the publishing data in the database or can set an invoke permission to the AWS IoT Core that invokes the Lambda function, and the lambda function will process the data and then store it in the database (AWS RDS database instance). To use the AWS RDS database, create a database and table structure in the Amazon RDS instance to store the incoming sensor data.
To fetch the stored data and display it on the dashboard one can use AWS API Gateway that invokes a Lambda function that has the RDS database configuration and can fetch required data from the database. The dashboard can be stored on AWS S3 bucket that has the feature to host a website. Also, can be hosted using AWS EC2 instance that creates a virtual server in the AWS Cloud.
The advantage of the dashboard is that one can use any kind on IoT device from health care wearables to Industrial IoT including Smart house IoT.
Use Case:
The above-mentioned figure depicts the use of AWS services to visualize the real-time data from IoT device on a Dashboard:
Components:
- AWS IoT Core
- AWS Lambda
- AWS RDS
- AWS API Gateway
- AWS S3
We have an IoT device (FSR – FSRs are sensors that allow you to detect physical pressure, squeezing and weight.) The FSR is used to detect whether a chair is occupied or not. Thus, it sends the data about the occupancy of the chair (FSR) and the battery life of the FSR. The FSR is configured as an AWS IoT Thing and is set to send real-time data to AWS IoT Core. The communication between the Thing and the AWS IoT Core is MQTT Protocol (MQTT (Message Queuing Telemetry Transport) is a lightweight, publish-subscribe based messaging protocol designed for resource-constrained devices and low-bandwidth, high-latency, or unreliable networks.).
The data sent by the FSR is published on a specific topic on AWS IoT Core. This helps in analysis the data format, frequency of the data publishing, and processing. An AWS Lambda function is set up that gets triggered when the data is published on specific topic of AWS IoT Core, and it contained the configurations of AWS RDS database to process the data and store in the database.
To fetch the data from database, AWS API Gateway endpoints are used that acts as the entry point for the dashboard to access the stored data. The dashboard cannot directly interact with the database, so we make request to API endpoint. When the request is made it triggers the lambda function that fetches data from the AWS RDS database.
To visualize the chair occupancy data, we created a dashboard that is hosted on AWS S3 (object storage service) that also has a feature to host static content like HTML, CSS and JavaScript files. The dashboard retrieves data from the API Gateway endpoint using appropriate API calls. The dashboard application uses the fetched sensor data and visualize it in form of charts and graphs whether the chair is occupied or not and about the battery life of the device.
Integration with AWS IoT Greengrass:
AWS IoT Greengrass is an open-source Internet of Things (IoT) edge runtime and cloud service that helps you build, deploy and manage IoT applications on your devices. AWS IoT Greengrass can be deployed on the local device (like raspberry pi) connected to IoT things. This acts as an edge runtime, allowing to run the Lambda function locally on the device itself.
Below are the steps to integrate the AWS IoT Greengrass to IoT Thing:
- Download AWS IoT Greengrass Software on local device.
- Develop a Lambda Function that runs on the Greengrass Core.
- Deploy the Lambda Function.
- Configure the Lambda Function.
Beyond Basics:
Apart from the above mentioned, the additional features that can be added are:
- Real-Time Alerts and Notifications: One can set up a trigger for Lambda function to send real-time alerts and notification based on specific readings. This can include SMS alerts, email notifications, or updates on the dashboard.
- History of the Data: One can also create an API call to fetch the history that is the stored data in the database.
- Auto-scaling Lambda Function: Configure the Lambda function to automatically scale based on the incoming data.
Beyond AWS:
As shown AWS is most popular Cloud Computing Platform, there are many other platforms that can be suitable for the visualization of Real-time data of IoT device. They are as follows:
- Microsoft Azure IoT Core
- Google Cloud Platform IoT Core
Conclusion:
The combination of AWS services such as AWS IoT Core, AWS Lambda, AWS RDS, AWS API Gateway, and AWS S3 we can create a platform (dashboard) for real-time data visualization. Sensor data can be collected from different IoT devices and processed and visualized on a Dashboard using graphs and charts. This can be used for better understanding for the data received and better decision-making. This helps to transfer raw data to give some sense of information.
References:
