Node is an open source development platform for executing javascript code at server-side.
It is useful for applications need persistent connection between server and browser.It is mostly used for the real time applications such as chats and web push notifications.
Hapi js is a rich and open source node js framework for building applications and services.It enables developers to focus on writing reusable logic instead of building infrastructure.
You can refer the files provided in this
https://github.com/KtreeOpenSource/NodejsExamples/tree/master/AsyncRead_LargeFiles_HapiJs
Step 1: Install Hapi js Framework
After running existing node js applications, install hapi js framework into it using below command npm i hapi
Step 2: Define Route using Hapi js
Define a route inside an controller like below.Define method,path which must be a string and in the config define handler which will handles the request and send response to the client.whenever any request touches the server you will find http methods,request url and request payloads etc.In this route payload contains the content coming from the request payload.You can restrict the file maxsize using maxBytes and all.
server.route({
method: ‘POST’,
path: ‘/api/dashboard’,
config: {
handler: async (req, res) => {
//
},
payload: {
output: ‘stream’,
parse: true,
allow: ‘multipart/form-data’,
maxBytes: 10 * 1024 * 1024 //10 mb,
}
}
});
Step 4: Reading large files using papa-parse
Before reading the files install papa-parse into your existing application using below command npm i papaparse.Inside the handler function read the files in the form of step and chunks using papa-parse.In order to read the files step by step means single row at once, you can use step function in the papaparse.Once reading rows in file is done,it will call the complete method, where you can return the data. Check the below example
Papa.parse("http://example.com/big.csv", {
download: true,
step: function (row) {
console.log("Row:", row.data);
},
complete: function () {
console.log("All done!");
}
});
Chunk is a callback function, identical to step. However, this function is executed after every chunk of the file is loaded and parsed rather than every row.Do not use both chunk and step callbacks together, the result will be undefined.
Papa.parse(fs.createReadStream(__dirname + relativePathToFile, ‘utf8’), {
delimiter: delimiter,
header: true,
skipEmptyLines: true,
beforeFirstChunk: function (chunk) {
return chunk;
},
chunk: function chunk(results, parser) {
}
});