Pending
Getting started with streaming your uploaded content is straightforward in Node js. In this example, we will be taking a look at how you can achieve this using request.js, a pretty nifty HTTP request utility that should be a must-have in your utility belt. That’s if you’re going to be using Node js for the long haul. For those of you who might be new to the concept of streams. The problem with loading a large file into memory is that you can actually run out of memory and cause your application to crash. If it’s a production application that could be a nightmare you would want to avoid that at all costs.
The beauty of streams is they help you work around this memory problem by acting on chunks of data hence allowing you to minimize your memory footprint. You can imagine a stream like a tap in your kitchen sink. Once a stream is opened, data flows in chunks from its origin to the process consuming it, in theory, you can stream unlimited amounts of data.
Before we get started coding, ensure you have Node.js installed.
Once Node has been installed, check if it's working as expected by opening up your terminal window and enter the following command
node -v
Create a directory named myapp, change to it and run npm init. Follow the on-screen instructions to give your package a default name, version, author, test script. Hit enter which preselects default options. Then install express as a dependency. Express js will be used to illustrate our server that will be responsible for handling multipart uploaded content
$ mkdir myapp && cd $_$ npm init$ npm install express --save
In order to illustrate the streaming in action, we create our client code which will perform uploading of our content we can name the file index.js.
touch index.js
Our client.js code leverages on the request module a wrapper around Node’s built-in http module. Hence we need to ensure we have that installed by running
npm install requests --save
Append below code to index.js which serves as the client code responsible for performing formData upload where my_file attribute is a link to a file on your local machine. The FormData object lets you compile a set of key/value pairs to send using http request. It is primarily intended for use in sending form data but can be used independently from forms in order to transmit keyed data. The transmitted data is in the same format that the form's submit() method would use to send the data if the form's encoding type were set to multipart/form-data.
var request = require('request');
var fs = require('fs');var formData = {
my_field: 'file',
my_file: fs.createReadStream('C:\\temp\\recording.mp4')
};
request.post({
url: 'http://localhost:5000/api/v1/uploads',
formData: formData
}, function optionalCallback(err, httpResponse, body) {
if (err) {
return console.error('upload failed:', err);
}
console.log('Upload successful! Server responded with:', body);
});
On the server side, we will use busboy npm package to actually do the stream of the uploaded content. The additional packages http-status-codes contains static HTTP codes and morgan is a logging tool that anyone who works with HTTP servers in Node.js should learn to use. morgan is a middleware that allows us to easily log requests, errors, and more to the console.
For our use case, it helps to log out all incoming HTTP requests.
npm install busboy --save
npm install http-status-codes --save
npm install morgan --save
The idea behind the server is a simulate a public-facing service that you need to upload multi-part content to. One thing you want to keep in mind is when using Busboy it only parses application/x-www-form-urlencoded and multipart/form-data requests. Create a new file called server.js
touch server.js
Append the following code below
var express = require('express'),
HttpStatus = require('http-status-codes'),
morgan = require('morgan'),
packageConfig = require('./package.json'),
path = require('path'),
fs = require('fs'),
Busboy = require('busboy');
express()
.use(morgan('combined'))
.get('/api/v1', function(req, res) {
res.status(HttpStatus.OK).json({
msg: 'OK',
service: 'File Server'
})
})
.post('/api/v1/uploads', function(req, res) {
var busboy = new Busboy({
headers: req.headers
});
console.log(req.headers);
busboy.on('file', function(fieldname, file, filename, encoding, mimetype) {
file.on('data', function(data) {
console.log('File [' + fieldname + '] got ' + data.length + ' bytes');
});
file.on('end', function() {
console.log('File [' + fieldname + '] Finished');
});
var saveTo = path.join(__dirname, 'dir', path.basename(filename));
var outStream = fs.createWriteStream(saveTo);
file.pipe(outStream);
});
busboy.on('finish', function() {
res.writeHead(HttpStatus.OK, {
'Connection': 'close'
});
res.end("That's all folks!");
});
return req.pipe(busboy);
}).listen(process.env.PORT || 5000, function() {
var address = this.address();
var service = packageConfig.name + ' version: ' + packageConfig.version + ' ';
console.log('%s Listening on %d', service, address.port);
});
You will need to run the client as a separate node app while the server runs on a separate node app.