The objective of this research work was to develop a new API application with a carefully planned plan. Waterfall development framework is used in this thesis as the thesis mainly focuses on practical API application development. Some recommendations also suggest that in future development, the application should be tested and improved to handle high-scale data across regions, and that more security strategies should focus on protecting the application API and its assets.
Company background
By collecting data and analyzing customer behavior, the company has a clearer awareness of customer reaction to its products and services. 1UP media Oy focuses on creating a high-quality and affordable online system that will provide solutions to collect and analyze reliable customer behavior data that can be used to improve user businesses.
Project background
Objectives
Project scopes and limitations
This section describes the Software Development Life Cycle (SDLC) as the foundation of the development methodology and the Waterfall Development model as the framework to develop the software.
Software Development Life Cycle (SDLC)
Development of the product: In this phase, the programming code is created according to DDS documents. Testing the product: This is the stage where the software is carefully tested using certain scenarios to find bugs to fix them. Based on the feedback, the organization will decide to officially release the product or make changes to suit the target audience.
Waterfall
API structure explanation
By looking at figure x we can see that within each function it contains an API route where the API gateway is open specifically for that function and managed within that function. It is the way Client-side can connect and communicate with back-end side using HTTP requests. Other components apply the same method; first part of API declares which components that API belongs to, then another parameter is structured according to the requirements of each call.
Any API starting with the "/user" parameter is checked in the user operation component, just like "/data_collection", "/device_management". From the API request protocol, we handle that request separately, create business logic for each call, create access to the database, and respond to the request. In each component's data management, we create business logic related to database activities, such as creating new, updating, replacing, or deleting objects in the database.
We use the Microsoft Azure Table storage SDK for Node JS to build the business logic.
API Call Cycle
From the start of the process, when the call is made, our backend starts executing by sending that request to the middleware part. One of the most common tasks is to check where this request is coming from, whether it is valid by checking the user's authentication token. If the call meets all of the middleware's strict requirements, the request will be passed to the API request handler.
Otherwise, we will send a response to that request with an error or error signal and the process will end at that point. When the request is valid, the next step is to handle the process by analyzing the requirements of that request. API handler can be thought of as the task manager for API, business logic is not stored here, it simply separates requirement and import functions from another place we call "service factory".
If it is valid, we will execute data communication logic to put new data into database. In this step, any error response neither from the database nor from our business logic will result in breaking the API call. In that case we will return the Request with Success status along with data packets.
Another case is when the API call type is "POST", "PUT", "DELETE", normally, we will reply with single status to let you know that the Request has been handled and executed.
Structuring API folder
Node_modules" folder is the auto-generated directory that contains all the "node package modules" developed by the community and used in the project. Package.json" is considered the configuration page of the project automatically created by node JS is when the project started.When we run node server we set "server.js" as the first file to run.
It declares which port the service is running on Azure table storage, which opens API port to all the services. Config.js” is where all global and specific variables are used in the project. Those variables control how the project works as well as the environment that runs it.
For example, in a particular feature that we named "Feature A" because it requires a specific Restful API, the developer will create a file named "featureA.API.js" directly in that "Feature A" folder. Another example is when connecting to a database, all database related functions like querying, updating, adding new data related to feature A are stored in a file named "featureA.SQL.js". The Utility" folder is where common utilities are stored for use in other functions.
Within the thesis scope, we are able to do only one part of testing which is unit testing without expanding into integration testing.
Technologies
- Server-side Web API
- Representational State Transfer (REST) API
- JavaScript programming language
- JavaScript best practices
- Scalability
- Maintainability
- Software security assurance (SSA)
- Node JS
- Express JS framework
- Azure
- Azure App Service
On the server side, JavaScript is widely used on top of the Node JS environment to interact with databases, design APIs, etc. To be able to build highly scalable Node JS project, loose coupling theory should be applied. Node JS is an open source, cross-platform, runtime environment that supports server-side application development.
Node JS modules are designed to integrate API design to reduce the complexity of web server side applications. The difference between Node JS and other web server side applications like PHP is that it is designed to be non-blocking. Commands in Node JS are executed in parallel and callback functions are used to handle success or error signals.
Node Package Management (NPM) is the Node JS package manager installed with Node JS. Express JS is a web application server framework built on top of Node JS. It also means that Express JS is a popular, unofficial default server framework for Node JS.
Since the project requirement is to create and develop an API using Node JS technology, and the framework must be licensed as open source, several popular frameworks can be chosen.
Development tools
- Jira software
- Brackets Text Editor
- POSTman
- Git and Gitlab
- Mocha Unit testing library
- JSDoc
Developers only have to deal with the logic of the software; deployment environments will be taken care of by Azure App Services. Developers just need to send source codes to App services using Git, then it will handle everything and deploy the product. Since Node JS does not require compilation steps like other frameworks, we only need text editor to program the software.
Brackets is a free, open source text editor focused primarily on web development that has a large amount of support for the JavaScript programming language. It is a cross-platform software, supporting Mac OSX, Microsoft Windows OS and Linux-based OS, such as Ubuntu… (Adobe). Brackets is also a good choice as other development team projects are developed using this text editor.
In addition, it is very useful to compare several versions of the source codes to better understand what is causing the system problems and who is responsible for it. It supports many useful features like issue tracking and project wiki. Issue tracking is a feature that helps developers be notified if issues occur during development, and those who create an issue in GitLab can see the status of the issue if it's resolved.
It is an engine that scans all JavaScript files in a specific folder or project to create documentation that displays well.
Development process
- General setup
- Creating Authentication and User management components
- Creating Data collection and Device management components
- Unit testing
- Documenting the module
- Deployment
From the beginning of the process we create User component where user can create user ID and password and it will be stored in the database. Any user information stored in the database is encrypted to protect the user's sensitive data from hackers and other employees in the company. This can be seen as a foundation for custom devices that will be integrated in the future.
In the future, when a custom device is connected to our platform, we will use these components as a basis to control, monitor the connection. It is very convenient because it ensures the stability of the API, in the future when we need to change some modules or business logics, we just need to run test commands to check if new changes affect the API result negatively. JSDoc will generate the documentation as an HTML page to display all my documentation as a directory to be searched in the future.
After fixing possible conflicts that appear briefly in the log report from Git, we push the API application to Azure and then we used POSTman to test the API function using real URL instead of "localhost". First is device registration and installation, the machine can connect to the API through the Internet. All information from the machine was sent to the API and the API saved data to the database.
Web portal can connect to API application to generate multiple reports based on customer demands as the end customer. Fortunately, it was already predicted, so we had a backup plan for that problem in the machine. Nevertheless, pilot version has some problems that should be improved or given more attention in the future.
Lumi platform
Lumi Architecture design
API request call cycle
POSTman test API locally
POSTman test API on cloud
