An ActiveStorage S3 Direct Uploader: Part 3: Upload Progress

We left off having a functional uploadzone that can handle multiple files and upload them to directly S3. You can drag and drop a file into the zone and it will upload to S3. You can also click the zone and select multiple files to upload. Once the file is uploaded, we inform our rails application that there is a new Track with an audio_file attached to it. It's pretty cool, look:

uploadzone

The problem is pretty obvious, there is absolutely no visual feedback in the UI letting us know what is happening or what works. In fact, looking at that, we don't even kno what happened. Let's fix.

Adding the Uploads to a List

Commit

Let's start by adding a list of files that are being uploaded. We'll add a div to div#uploads for each file that is being uploaded,

At what point in our code do we know that the user wants to upload a file so that we can grab it and put the file to be uploaded into div#uploads?

If you open up app/javascript/controllers/uploadzone_controller.js you'll see that we have a uploadFile method that gets called to upload a file. Let's modify that so that before the upload starts, we can insert a div with the file name. After the DirectUpload instance is created but before we call create on it to upload it, we'll insert our code.

app/javascript/controllers/uploadzone_controller.js

uploadFile(file) {
  const upload = new DirectUpload(file, "/rails/active_storage/direct_uploads");

  this.insertUpload(upload);

  upload.create((error, blob) => {
    // Rest of upload logic
  });
}

Because there's going to be a good amount of logic to insert the element into the dom, instead of implementing it there, we are just going to call the non-existent method insertUpload. We pass it the instance of upload so that it has access to the upload.file.

Now we need to write the insertUpload method.

app/javascript/controllers/uploadzone_controller.js

insertUpload(upload) {
  const fileUpload = document.createElement("div");
  fileUpload.textContent = upload.file.name;

  const uploadList = document.querySelector("#uploads");
  uploadList.appendChild(fileUpload);
}

If you try an upload now, you should see:

uploads

So we are getting some visual feedback. The next thing we want to do, in addition to styling these elements a bit, is add a div.progress element to the div so that we can update it with the file progress as the file is being uploaded. Most of this is just tailwind classes, but the important thing to pay attention to is the id of the div for the fileUpload.

app/javascript/controllers/uploadzone_controller.js

  insertUpload(upload) {
    const fileUpload = document.createElement("div");

    fileUpload.id = `upload_${upload.id}`;
    fileUpload.className = "p-3 border-b";

    fileUpload.textContent = upload.file.name;

    const progressWrapper = document.createElement("div");
    progressWrapper.className = "relative h-4 overflow-hidden rounded-full bg-secondary w-[100%]";
    fileUpload.appendChild(progressWrapper);

    const progressBar = document.createElement("div");
    progressBar.className = "progress h-full w-full flex-1 bg-primary";
    progressBar.style = "transform: translateX(-100%);";
    progressWrapper.appendChild(progressBar);

    const uploadList = document.querySelector("#uploads");
    uploadList.appendChild(fileUpload);
  }

The most important part is setting the id of the fieUpload. Each upload instance gets an id, which is the index of the upload (so if we were uploading 4 files, the second one would be 2). By doing this, as we are getting progress for the upload, we can find this div easily again to update the .progress element.

If we upload a few files now, the UI looks way better.

uploads

Updating the Progress Bar

If you look deep in the ActiveStorage documentation, you will find a section for tracking the progress of a file upload. We're going to use that.

Basically, if we pass DirectUpload a third argument, a context, DirectUpload can attach the xhr request to it and trigger a callback, directUploadWillStoreFileWithXHR on the context passing it the request object of the xhr request.

This part gets confusing, so hang on.

In order to track the individual progress of each upload, we need to represent each upload as an instance of something that can have its own scope so that we can use that as the context for the DirectUpload instance. Without this, it would be impossible to know which xhr request relates to which upload.

The good news is that even if you don't quite get that, and by the way, the first 3 times I built this, I didn't get that and just took code samples for granted, the code we're going to implement is a logical refactor anyway.

What we're going to do is extract all this upload logic, which is now a lot, out of our controller and into a model of sorts, an Upload class. Every time we want to upload a file, we will instantiate an instance of Upload and let that object do all the work. Our controller will be lean and clean and our Upload class will be responsible for all the upload logic.

Extracting Upload

Normally I would put this class in app/javascript/models/Upload.js but because that would mean having to import it into the controller, which means another interaction with importmaps which is confusing enough, I'm just going to put it in app/javascript/controllers/uploadzone_controller.js for now. We can always move it later.

At the top of app/javascript/controllers/uploadzone_controller.js add:

import { Controller } from "@hotwired/stimulus";
import { DirectUpload } from "@rails/activestorage";
import { post } from "@rails/request.js";

class Upload{

}

export default class extends Controller {

Now we've got a place to define this class and the controller will have access to it because it's in the same file.

The extraction point, the point at which we want to pass control from the controller to the Upload class is in the files loop of acceptFiles. Instead of calling this.uploadFile(file) we'll call new Upload(file) and then some sort of method to basically say do your thing or more programmatically process().

Let's define a constructor for Upload and stub out a process method, and make this cut so that we can then move over the rest of the upload logic out of the controller and into this instance.

class Upload {
  constructor(file) {
    // Here's where we will instantiate the DirectUpload instance.
  }

  process() {
    // Here's where we will actually kick off the upload process.
  }
}

We're also going to make the refactor cut and instead of directly calling this.uploadFile in acceptFiles, we're going to instantiate an instance of Upload and call process on it.

acceptFiles(event) {
  event.preventDefault();
  const files = event.dataTransfer ? event.dataTransfer.files : event.target.files;
  [...files].forEach((f) => {
    new Upload(f).process();
  });
}

It's nice to test out that this is working by either putting a console.log or a debugger in the process method and making sure it's being called once per file.

Extract Class/Method

The basic refactoring technique we're using here is Extract Class and Extract Method. We've already extracted our class, and now we want to extract all the methods relating to upload into this class.

What's really nice here is that we're basically following the Model/Controller paradigm of MVC in our Stimulus controller.

The controller itself is responsible for the controller logic, taking things from the view, the dropzone, and passing them to the model.

Much like it takes a second in Rails to realize you can just introduce POROs (Plain Old Ruby Objects) wherever you want and maintain really nice Object Orientation, in Stimulus, your controller can and probably should, interact with other well encapsulated POJOs (Plain Old JavaScript Objects). This is a really nice way to keep your code clean and organized and makes testing so much easier.

Implementing Upload

Let's just move over one method at a time and update its variables and scope to maintain the current functionality. Let's start with uploadFile.

class Upload {
  constructor(file) {
    // Here's where we will instantiate the DirectUpload instance.
  }

  process() {
    // Here's where we will actually kick off the upload process.
    console.log(this);
  }

  uploadFile(file) {
    const upload = new DirectUpload(file, "/rails/active_storage/direct_uploads");

    this.insertUpload(upload);

    upload.create((error, blob) => {
      if (error) {
        // Handle the error
      } else {
        const trackData = { track: { filename: blob.filename }, signed_blob_id: blob.signed_id };

        post("/tracks", {
          body: trackData,
          contentType: "application/json",
          responseKind: "json",
        });
      }
    });
  }
}

Looking at this, uploadFile basically looks like it should be our process implementation. So we're going to move all the logic into there.

We're also going to move the DirectUpload instance into the constructor so that we can make the instance an instance property and then access it from process.

We're doing all of this after all so that instances can properly maintain their own scope and data.

class Upload {
  constructor(file) {
    this.directUpload = new DirectUpload(file, "/rails/active_storage/direct_uploads");
  }

  process() {
    //this.insertUpload(upload);

    this.directUpload.create((error, blob) => {
      if (error) {
        // Handle the error
      } else {
        const trackData = { track: { filename: blob.filename }, signed_blob_id: blob.signed_id };

        post("/tracks", {
          body: trackData,
          contentType: "application/json",
          responseKind: "json",
        });
      }
    });
  }
}

Way better. This should be functional. Try it out in your browser and see if you still see the upload requests firing.

Let's move on to insertUpload. Extracting that is mostly about identifying out of scope references and correcting them. There doesn't look to be any as long as we're passing in this.directUpload as an argument. from process. If we make that change, we should see the method work as expected. It does.

But there's no reason to pass that argument in as it is a property of the instance so we can just update the method to use this.directUpload directly and remove the argument from the method call in process.

insertUpload() {
  const fileUpload = document.createElement("div");

  fileUpload.id = `upload_${this.directUpload.id}`;
  fileUpload.className = "p-3 border-b";

  fileUpload.textContent = this.directUpload.file.name;

  const progressWrapper = document.createElement("div");
  progressWrapper.className = "relative h-4 overflow-hidden rounded-full bg-secondary w-[100%]";
  fileUpload.appendChild(progressWrapper);

  const progressBar = document.createElement("div");
  progressBar.className = "progress h-full w-full flex-1 bg-primary";
  progressBar.style = "transform: translateX(-100%);";
  progressWrapper.appendChild(progressBar);

  const uploadList = document.querySelector("#uploads");
  uploadList.appendChild(fileUpload);
}

And that's it! We've entirely extracted the upload logic out of the controller and into a separate class. We're now at the point where we can implement the progress bar.

Implementing the Progress Bar

Now that we have a context that represents the indiviual uploads we can pass that context, which is the instance of the Upload itself into DirectUpload.

constructor(file) {
  this.directUpload = new DirectUpload(file, "/rails/active_storage/direct_uploads", this);
}

What this means is that DirectUpload can now call methods on the instance of Upload that it's been passed. This is how we're going to implement the progress bar. Specifically, we're going to implement the directUploadWillStoreFileWithXHR method. This is the hook that DirectUpload provides and passes the xhr request object too. Once we have that object, we can add an event listener to it to be notified of the progress of the upload.

Within the Upload class, continue by defining:

directUploadWillStoreFileWithXHR(request) {
  request.upload.addEventListener("progress", (event) => this.updateProgress(event));
}

This method gets automatically called by DirectUpload during create as it actually conducts the upload. We listen to the progress event on the request.upload object and call updateProgress with the event.

For now, let's implement updateProgress to just log the event and the two properties we care about, event.loaded and event.total.

updateProgress(event) {
  console.log(event.loaded, event.total);
}

Logging Progress

This is great. We're getting the progress of the upload. Now we just need to update the progress bar. We're going to convert the loaded and the total to percentages and then update the progress bar translateX value to move the black div to the right, representing progress.

updateProgress(event) {
  const percentage = (event.loaded / event.total) * 100;
  const progress = document.querySelector(`#upload_${this.directUpload.id} .progress`);
  progress.style.transform = `translateX(-${100 - percentage}%)`;
}

Who remembers why #upload_${this.directUpload.id} .progress finds the correct elements for this upload's progress to update?

It's because in insertUpload when we add the upload to the DOM, we set the id of the upload to upload_${this.directUpload.id}. This means that we can find the upload by its id and then find the progress bar within it.

This was the entire point of the Upload refactor. To be able to maintain the scope of each individual upload. Basically, to encapsulate the upload into an instance that could then later refer to itself and find it's progress bar again.

With that code, we're done.

Final Product

Here's the entire app/javascript/controllers/uploadzone_controller.js file:

import { Controller } from "@hotwired/stimulus";
import { DirectUpload } from "@rails/activestorage";
import { post } from "@rails/request.js";

class Upload {
  constructor(file) {
    this.directUpload = new DirectUpload(file, "/rails/active_storage/direct_uploads", this);
  }

  process() {
    this.insertUpload();

    this.directUpload.create((error, blob) => {
      if (error) {
        // Handle the error
      } else {
        const trackData = { track: { filename: blob.filename }, signed_blob_id: blob.signed_id };

        post("/tracks", {
          body: trackData,
          contentType: "application/json",
          responseKind: "json",
        });
      }
    });
  }

  insertUpload() {
    const fileUpload = document.createElement("div");

    fileUpload.id = `upload_${this.directUpload.id}`;
    fileUpload.className = "p-3 border-b";

    fileUpload.textContent = this.directUpload.file.name;

    const progressWrapper = document.createElement("div");
    progressWrapper.className = "relative h-4 overflow-hidden rounded-full bg-secondary w-[100%]";
    fileUpload.appendChild(progressWrapper);

    const progressBar = document.createElement("div");
    progressBar.className = "progress h-full w-full flex-1 bg-primary";
    progressBar.style = "transform: translateX(-100%);";
    progressWrapper.appendChild(progressBar);

    const uploadList = document.querySelector("#uploads");
    uploadList.appendChild(fileUpload);
  }

  directUploadWillStoreFileWithXHR(request) {
    request.upload.addEventListener("progress", (event) => this.updateProgress(event));
  }

  updateProgress(event) {
    const percentage = (event.loaded / event.total) * 100;
    const progress = document.querySelector(`#upload_${this.directUpload.id} .progress`);
    progress.style.transform = `translateX(-${100 - percentage}%)`;
  }
}

export default class extends Controller {
  static targets = ["fileInput"];
  connect() {
    this.element.addEventListener("dragover", this.preventDragDefaults);
    this.element.addEventListener("dragenter", this.preventDragDefaults);
  }

  disconnect() {
    this.element.removeEventListener("dragover", this.preventDragDefaults);
    this.element.removeEventListener("dragenter", this.preventDragDefaults);
  }

  preventDragDefaults(e) {
    e.preventDefault();
    e.stopPropagation();
  }

  trigger() {
    this.fileInputTarget.click();
  }

  acceptFiles(event) {
    event.preventDefault();
    const files = event.dataTransfer ? event.dataTransfer.files : event.target.files;
    [...files].forEach((f) => {
      new Upload(f).process();
    });
  }
}

Recap and Next Steps

This post covered a lot! Let's summarize the major steps and why we took them.

  1. First we added a representation of the upload into the DOM. This was the insertUpload method.

  2. Upon building that, we realized we couldn't refer to each individual upload because we didn't have a way to scope it. So we created the Upload class to encapsulate each upload.

  3. We moved all the upload related methods from the controller into the Upload class. We updated any scope issues.

  4. We sent the upload instance to DirectUpload so that it could call methods on it.

  5. We then added the directUploadWillStoreFileWithXHR method to the Upload class to be able to listen to the progress of the upload.

  6. Finally, we added the updateProgress method to update the progress bar.

So it was a lot.

I'm going to write one more quick post tomorrow on using the response from Rails to update the upload with the track's name and link to it and embed a small music player. Then we'll be done with this series and you'll have built a pretty awesome drag and drop upload experience.

Did you find this article valuable?

Support Avi Flombaum by becoming a sponsor. Any amount is appreciated!