How to Upload a File to Amazon S3 with NodeJS

As a software engineer you may need to upload files to Amazon S3 for storage backup or sharing purposes Amazon S3 Simple Storage Service is a highly scalable secure and durable cloud storage service that allows you to store and retrieve any amount of data from anywhere on the web In this tutorial we will show you how to upload a file to Amazon S3 using NodeJS a popular server-side JavaScript runtime environment.

Prerequisites

Before we start, you should have the following:

• An AWS account with access keys.
• NodeJS and NPM installed on your local machine.
• A basic understanding of JavaScript, NodeJS, and AWS S3.

Step 1: Set up an S3 Bucket

The first step is to create an S3 bucket in your AWS account. An S3 bucket is a container for storing objects, such as files, in the Amazon S3 cloud. To create an S3 bucket, follow these steps:

1. Log in to your AWS account and navigate to the S3 console.
2. Click on the “Create bucket” button.
3. Enter a unique name for your bucket, select a region, and choose the default settings for versioning, logging, and permissions.
4. Click on the “Create bucket” button to create the bucket.

Once you have created an S3 bucket, you can upload files to it using NodeJS.

Step 2: Install the AWS SDK for NodeJS

To interact with AWS services using NodeJS, you need to install the AWS SDK for NodeJS. The AWS SDK provides a set of JavaScript libraries for accessing and managing AWS services, including Amazon S3. To install the AWS SDK for NodeJS, run the following command in your terminal:

npm install aws-sdk

Step 3: Set up AWS Credentials

To access your AWS account programmatically, you need to provide your access keys to the AWS SDK. You can obtain your access keys from the AWS console by following these steps:

1. Log in to your AWS account and navigate to the IAM (Identity and Access Management) console.
2. Click on the “Users” tab and select your IAM user.
3. Click on the “Security credentials” tab and expand the “Access keys” section.
4. Click on the “Create access key” button to generate a new access key pair.
5. Download the access key file and keep it in a safe place.

Once you have obtained your access keys, you can set them up in your NodeJS code using the AWS SDK. The easiest way to do this is to set them up as environment variables, like this:

process.env.AWS_ACCESS_KEY_ID = 'your-access-key-id'
process.env.AWS_SECRET_ACCESS_KEY = 'your-secret-access-key'

Alternatively, you can set them up programmatically in your NodeJS code, like this:

const AWS = require('aws-sdk');

const s3 = new AWS.S3({
  accessKeyId: 'your-access-key-id',
  secretAccessKey: 'your-secret-access-key'
});

Step 4: Upload a File to S3

Now that you have set up your S3 bucket and AWS credentials, you can upload a file to S3 using NodeJS. To do this, you need to use the AWS SDK’s S3 API to create a new object in your S3 bucket and upload the file data to it. Here is an example NodeJS code that uploads a file to S3:

const AWS = require('aws-sdk');
const fs = require('fs');

const s3 = new AWS.S3();

const bucketName = 'your-bucket-name';
const fileName = 'path/to/your/file';
const fileData = fs.readFileSync(fileName);

s3.upload({
  Bucket: bucketName,
  Key: fileName,
  Body: fileData
}, (err, data) => {
  if (err) {
    console.error(err);
  } else {
    console.log(`File uploaded successfully. ${data.Location}`);
  }
});

In this code, we first import the AWS SDK and the NodeJS built-in fs module for file system operations. We then create a new instance of the AWS.S3 class, which represents the S3 service. We specify the name of our S3 bucket, the path to our local file, and read the file data using the fs.readFileSync() method. Finally, we call the s3.upload() method with the bucket name, file key (path), and file data as parameters. We also provide a callback function to handle the upload result, which logs the file URL if the upload is successful.

Complete Source Code

calling upload File function

const fs =  require('fs');
const path = require("path");
const { v4: uuidv4 } = require("uuid");
const AWS = require('aws-sdk');

const s3 = new AWS.S3({
  accessKeyId: process.env.AWS_ACCESS_KEY_ID,
  secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
});

exports.uploadfile = function(data, file_name) {
    return new Promise((resolve, reject) => {
      let dir_name = process.env.UPLOAD_PATH;
      if (!fs.existsSync(dir_name)){
        fs.mkdirSync(dir_name);
      }
      let unique = uuidv4();
      let fileName  = unique + path.extname(file_name);
     var newPath = `${dir_name + fileName}`;
      fs.writeFile(newPath, data, function (err) {
        if (err) return reject(err) ;
        const fileData = fs.readFileSync(newPath);
        s3.upload({
          Bucket: process.env.AWS_BUCKET_NAME,
          Key: fileName,
          Body: fileData
        }, (err, data) => {
          if (err) {
            return reject(err)
          } else {
             console.log('File uploaded successfully.',data);
            fs.unlinkSync(newPath) // delete temp file after upload
            return resolve(data.key) // return filename
          }
        });
      });
    })
}

Conclusion

In this tutorial, we have shown you how to upload a file to Amazon S3 using NodeJS. We first set up an S3 bucket and obtained our AWS access keys. We then installed the AWS SDK for NodeJS and set up our AWS credentials. Finally, we wrote a sample NodeJS code that uploads a file to S3 using the AWS SDK’s S3 API. With this knowledge, you can now integrate Amazon S3 into your NodeJS applications for storing and retrieving files in the cloud.