How to Add Webcam Photo Filters & Effects in React with Cloudinary

What are photo filters?

If you’ve used any of the wildly popular apps mentioned before, you might have seen the ability to snap a photo and add some kind of transformation to it, whether that’s making that photo look more flattering or simply changing some of the color.

A lot of these apps also let you add these filters in realtime (depending on the filter and app), which gives you some super fun ways to interact in the app.

But what we’re looking for is that layer that goes on top of the original photo, the filter, which transforms the photo into something different.

How can Cloudinary add photo filters?

Cloudinary has a wide range of transformations they can apply to media, including everything from optimization to color effects, all the way to background removal and neural artwork style transfers.

Cloudinary can do a lot of these transformations in two different ways, when being uploaded or on the fly. When it’s stored, or when being delivered before caching it, Cloudinary will apply these effects with it’s algorithms or external plugins so that it’s made available to deliver right in the browser.

For our purposes, we’re interested in the on-the-fly transformations, as when we grab someone’s photo with their webcam, we can apply these transformations based on what they select.

What are we going to build?

We’re going to build an app that takes advantage of a device’s webcam to capture the image and apply photo effects and filters.

To do that, we’ll use a starter I created that bootstraps some of the UI that we’ll use in the walkthrough.

We’ll then add the ability to capture the webcam and grab a snapshot using a React library that uses the native browser’s webcam capabilities.

With that photo, we’ll plug it into Cloudinary by uploading it to our media library, then we’ll use Cloudinary features and custom mixes of features to add filters to our photos.

Step 0: Creating a new Next.js app from a demo starter

We’re going to start off with a new Next.js app using a starter that includes some simple UI that we’ll use.

Inside of your terminal, run:

yarn create next-app my-camera-filters -e https://github.com/colbyfayock/demo-camera-filters-starter
# or
npx create-next-app my-camera-filters -e https://github.com/colbyfayock/demo-camera-filters-starter

Note: feel free to use a different value than my-camera-filters as your project name!

Once installation has finished, you can navigate to that directory and start up your development server:

cd my-camera-filters

yarn dev
# or
npm run dev

And once loaded, you should now be able to open up your new app at http://localhost:3000!

At this point, if you take a second to poke around the app, you’ll see we have a few key things set up.

We’ll see a still image of a mountain, which is where we’ll be adding our webcam image and eventually the preview with filters applied.

We also have some action buttons along with a thumbnail where we’ll add some filter options that people can select when using our app.

Step 1: Setting up a webcam viewer in a Next.js app

Starting off, let’s update our mountain photo and replace it with an actual webcam.

To do this, we’ll use React Webcam, which takes advantage of the browser’s MediaDevices API.

First, let’s install the package in our project.

In your terminal run:

yarn add react-webcam
# or
npm install react-webcam

Next, we can import the dependency into our app.

At the top of src/pages/index.js add:

import Webcam from 'react-webcam';

With our Webcam module imported, we can start using it.

Ultimately, we want to use Webcam as a React component, but we want to add a little configuration on top of that. Particularly, we want to add sizing and “constrains” which basically define the viewport of the webcam that we’re using.

Above the Home component definition, let’s add our constraints as a constant:

const cameraWidth = 720;
const cameraHeight = 720;
const aspectRatio = cameraWidth / cameraHeight;

const videoConstraints = {
  width: {
    min: cameraWidth
  },
  height: {
    min: cameraHeight
  },
  aspectRatio
};

We’re stating that we want to set up a webcam with a definition of 720×720 and aspect ratio based off of those values, which will be 1:1, meaning, a square. While it’s not critical to set those values up as constants, this helps make it easy to configure should you want to change the size.

But now, let’s use these values and add our Webcam component.

Replace the img tag that we’re using to show a mountain with:

<Webcam videoConstraints={videoConstraints} width={cameraWidth} height={cameraHeight} />

And just like that, once we reload the page and click Allow when your browser asks to give webcam permission, we can now see ourselves in the camera!

Next, we’ll learn how we can capture a still of our webcam and replace our live feed with it to prepare for filter effects.

Follow along with the commit!

Step 2: Capturing a still of a webcam view in React

Given the React Webcam component uses the native browser APIs in order to capture the webcam view, we need to access that component’s internal reference so that we can interact with it using native APIs.

To do this, we’ll first set up a ref for our component.

First, let’s import the useRef hook from React.

At the top of src/pages/index.js add:

import { useRef } from 'react';

Next, we want to define our ref.

At the top of the Home function component, add:

const webcamRef = useRef();

Then we need to set that ref on our webcam component.

Update the Webcam instance to:

<Webcam ref={webcamRef} videoConstraints={videoConstraints} width={cameraWidth} height={cameraHeight} />

At this point we won’t notice any differences, but now we can set up an event handler so that we can access our camera and take a snapshot.

To do this we’re going to use React state and the webcam’s screenshot method to take a screenshot and then store it in state so that we can use it in our app.

First let’s import the React useState hook so that we can set up state. We can do this by appending the import to our existing one with useRef:

import { useRef, useState } from 'react';

Then we can create an instance of state where we can store our image data.

Inside of the Home component add:

const [image, setImage] = useState();

And then we can set up our function to actually capture our screenshot:

function handleCaptureScreenshot() {
  const imageSrc = webcamRef.current.getScreenshot();
  setImage(imageSrc);
}

Here we’re accessing our webcam ref and using the current value of it to trigger the getScreenshot method. That will return image data which we then store in state using the setState function.

Before we can use it, we need to wire it up to the app.

On the Capture Photo button, let’s add a click handler:

<Button onClick={handleCaptureScreenshot}>
  Capture photo
</Button>

Now finally, let’s show our image!

Instead of always showing our webcam viewer, we’re going to only show it if we haven’t yet taken a screenshot. If we have taken a screenshot, we’ll show that.

Let’s update our Webcam component block to:

<div className={styles.stage}>
  {image && (
    <img src={image} />
  )}
  {!image && (
    <Webcam ref={webcamRef} videoConstraints={videoConstraints} width={cameraWidth} height={cameraHeight} />
  )}
</div>

Here we’re checking to see if our image value exists and displaying an image or the Webcam component depending on if it does.

And when we refresh the page, we can click the Capture Photo button and instead of the live view, we now have a screenshot of our webcam view!

We can even wire up the Reset button so that if we don’t like how it turned out, we can try to take another shot.

Update the Reset button to:

<Button onClick={() => setImage(undefined)} color="red">
  Reset
</Button>

And now you can capture a photo and reset until you find what capture one you like!

Follow along with the commit!

Step 3: Uploading images to Cloudinary in a serverless function with the Cloudinary Node SDK

Now that we have our image data available, we want to set it up so that we can use it with Cloudinary to add photo filters.

Particularly, we want to upload that image to our Cloudinary media library where we’ll then display that photo and later use it for transformations. We’ll do that by using a serverless function via an API endpoint where we can process the request.

To start, we need to install the Cloudinary Node SDK.

In your terminal run:

yarn add cloudinary
# or
npm install cloudinary

When using this particular SDK, we can’t use it inside of React, as it’s designed to use native Node APIs.

So instead, we’re going to create a serverless function.

Inside of the src/pages/api directory, let’s create a new file called upload.js.

Then inside of src/pages/api/upload.js add:

export default async function handler(req, res) {
  const { image } = JSON.parse(req.body);
  res.status(200).json({
    test: true
  });
}

Here we’re defining the handler for our endpoint’s serverless function and inside, parsing the image property, which we’ll post inside of our app.

Next let’s use our new endpoint in our app.

We’re going to use the React useEffect hook, so that any time that image data changes, we upload it to Cloudinary.

First let’s import it with the rest of our React hooks:

import { useRef, useState, useEffect } from 'react';

Then, we’ll use that hook inside of the Home component:

useEffect(() => {
  if ( !image ) return;

  (async function run() {
    const response = await fetch('/api/upload', {
      method: 'POST',
      body: JSON.stringify({
        image
      })
    }).then(r => r.json());
    console.log('response', response);
  })();
}, [image]);

Here, the first time the component loads and any time the image value changes, the function inside of our useEffect will run.

If we don’t have any image data, we return immediately, as we don’t need to do anything with it, but if we do have image data, we post it to our new upload endpoint.

We can see this working in action by opening our app and clicking the Capture button, which will tigger the effect, post to the API, and log the response, where we see our test object.

So now let’s actually do something with that image data.

To make this work, we’re going to take the image data that we’re receiving in the serverless function, send it off to Cloudinary, and back in our app, store the results in a new state instance so that we can display it on the page.

First off, let’s configure the Cloudinary SDK in our serverless function.

At the top of src/pages/api/upload.js let’s first import v2 of the SDK with:

const cloudinary = require('cloudinary').v2;

Then let’s configure the SDK with our account:

cloudinary.config({
  cloud_name: process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME,
  api_key: process.env.CLOUDINARY_API_KEY,
  api_secret: process.env.CLOUDINARY_API_SECRET,
  secure: true
});

Here you’ll notice that we’re passing environment variables to the Cloudinary configuration method.

Most of these values are sensitive values, meaning, we don’t want them exposed or stored in Git, so we use environment variables to keep them safe.

That also means we need to set up an enviornment variable file.

Create a new file in the root of your project called .env.local and inside add:

NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME="<Your Cloud Name>"
CLOUDINARY_API_KEY="<Your Cloudinary API Key>"
CLOUDINARY_API_SECRET="<Your Cloudinary API Secret>"

Replace each of these values with your own values from your Cloudinary account.

Note: the NEXT_PUBLIC_ prefix for our Cloud Name tells Next.js that we want to use that value clientside, where the rest can only be used serverside. We’ll be using these values again in later steps, hence adding the prefix now.

You can find them all right at the top of the dashboard when logged into cloudinary.com.

But now that we have our account configured, we can upload our image data right to Cloudinary.

Inside of the handler function inside src/pages/api/upload.js add:

const results = await cloudinary.uploader.upload(image, {
  folder: 'cloudinary-camera-filters'
});

Note: feel free to update the folder to be wherever you want or remove it to store the images in the root of your media library.

And once that request is done, we can return those results instead of our test object:

res.status(200).json({
  ...results
});

Because we’re already logging the response from our useEffect hook, we can head right into our app and try this out.

We can see the full response from Cloudinary including the URL to our newly uploaded image!

So finally, let’s display that image in our app as soon as it’s done uploading.

To do that, we’re going to store that full data object as state, then display it when it’s available.

At the top of the Home component in src/pages/index.js add:

const [cldData, setCldData] = useState();

Next, we’ll take the response of our useEffect and fire our setCldData function with it:

const response = await fetch('/api/upload', {
  method: 'POST',
  body: JSON.stringify({
    image
  })
}).then(r => r.json());

setCldData(response);

And finally, we’ll use that to display in our app once it loads.

Update the image tag that’s currently showing our image data with:

{ image && (
  <img src={cldData?.secure_url || image} />
)}

If we have the data and have taken it to the next step of successfully uploading it, we want to use our newly available Cloudinary URL.

And now, once we capture our image and wait a second or two, we should see a successful network request in our console and no changes to the image appearance itself.

Once that request does finish, you can even right click the image and open in a new tab, where you’ll see your Cloudinary URL load.

We uploaded our image data to Cloudinary, so once we swap our Cloudinary URL, it should look exactly the same right now, but we’ll see in the next steps how we can now add filter effects with Cloudinary.

Follow along with the commit!

Step 4: Delivering Cloudinary images clientside with the Cloudinary URL Gen SDK for JavaScript

In the previous step, we used the Cloudinary Node SDK in order to upload our image data to our media library.

The Cloudinary Node SDK is fantastic, but it’s intended to be used strictly in node.

While we could technically try to process all of our images and transformations in node so that we can use the Node SDK, it’s not as straightforward and we lose flexibility when trying to perform actions clientside.

So instead, we’re going to additionally install and use the Cloudinary URL Gen SDK for JavaScript so that we can configure our image URLs in our app and later add transformations.

To start, let’s install the URL Gen SDK:

yarn add @cloudinary/url-gen 
# or
npm install @cloudinary/url-gen

Then we want to import the package to our app.

Inside src/pages/index.js add:

import { Cloudinary } from '@cloudinary/url-gen';

To use the URL Gen SDK, we need to configure it just like we did the Node SDK, though this time, we only need to pass our Cloud Name.

Above our Home component definition add:

const cloudinary = new Cloudinary({
  cloud: {
    cloudName: process.env.NEXT_PUBLIC_CLOUDINARY_CLOUD_NAME
  },
  url: {
    secure: true
  }
});

We’re creating a new instance of the Cloudinary URL Gen SDK where we’re passing the same Cloudinary Cloud Name environment variable that we created in the last step.

But now, we can use this SDK to change how we deliver our images from Cloudinary.

To start, let’s create some new variables to help us manage using both our raw image data and Cloudinary URL.

Underneath our state instances add inside of the Home component:

let src = image;
const cldImage = cldData && cloudinary.image(cldData.public_id);

if ( cldImage ) {
  src = cldImage.toURL();
}

Here we’re defining a new variable src that we’ll use to store any image that we want to send to the app whether the raw image data or from Cloudinary.

We’re also creating a new constant called cldImage where first, we check to see if we have any data inside of cldData (where we store a successful upload) and if we do, we use the image method of the Cloudinary SDK to create a new image instance.

You’ll notice that we’re passing in the public ID of cldData. Because we’re using the Cloudinary SDK, we can pass in the public ID, which is the identifier Cloudinary uses for media, which will allow us to configure new URLs.

Finally, if cldImage exists, we use the toURL method to grab the Cloudinary image URL and set our src variable with that value.

To use this value, let’s update our image and webcam instance:

{src && (
  <img src={src} />
)}
{!src && (
  <Webcam ref={webcamRef} videoConstraints={videoConstraints} width={cameraWidth} height={cameraHeight} />
)}

Instead of checking if the image value exists, we check if src exists, which could be either the raw image or the Cloudinary URL.

But now if we open up the app and run our usual test of capturing a photo, we’ll notice that it’s still completely the same!

The only difference that you might notice if you caught it is our Cloudinary URL might look slightly different as now it’s being put together by the SDK instead of returned from the API.

But not seeing any changes is a good sign, as we want our SDK to deliver our image consistent with the image itself.

In the next step, we’ll learn how we can start applying transformations with our Cloudinary URL Gen SDK and start adding some filters!

Follow along with the commit!

Step 5: Adding filters to a webcam image with Cloudinary

Now that we have our image captured and uploaded to Cloudinary, we can start taking advantage of adding effects and filters.

While Cloudinary has a ton of different transformations we can apply, we’re going to use the art filters which allow us to easily add Instagram-like filter effects to our photos.

To see how this works, we can try it out on our main image.

Inside of the if statement block where we turn our cldImage into a URL, let’s add a transformation.

Update the if statement contents to:

if ( cldImage ) {
  cldImage.effect(`e_art:incognito`);
  src = cldImage.toURL();
}

Here we’re using the effect method and passing in the effect we want to use to apply a the Incognito art filter.

There’s a large list of different art filters we can use, such as if we wanted to try another one like eucalyptus we can update our code to:

cldImage.effect(`e_art:eucalyptus`);

And you can see we get a different effect.

The nice thing is we can easily swap out any of those values, so instead of doing that manually, we can add all of those filters as items in our list at the bottom, giving someone the ability to change it to find the one they like.

To start, let’s grab the full list of art filters available. You can find that in the Cloudinary docs at https://cloudinary.com/documentation/transformation_reference#e_art.

And let’s add a constant to the top of our file that includes all of those values:

const ART_FILTERS = [
  'al_dente',
  'athena',
  'audrey',
  'aurora',
  'daguerre',
  'eucalyptus',
  'fes',
  'frost',
  'hairspray',
  'hokusai',
  'incognito',
  'linen',
  'peacock',
  'primavera',
  'quartz',
  'red_rock',
  'refresh',
  'sizzle',
  'sonnet',
  'ukulele',
  'zorro'
];

Next, let’s set up our filters list to map through each one.

Under the Filters header, replace the unordered list (ul) with:

<ul className={styles.filters}>
  {ART_FILTERS.map(filter => {
    return (
      <li key={filter} data-is-active-filter={false}>
        <button className={styles.filterThumb}>
          <img width="100" height="100" src="/images/mountain-100x100.jpg" alt="Filter Name" />
          <span>{ filter }</span>
        </button>
      </li>
    )
  })}
</ul>

Here we’re mapping through each filter in our array and adding a list item for that filter. We’re also using that value to show the name of the filter.

And if we look at our list of filters at the bottom of the page, we should see them all available.

With our filters, we need to allow someone to choose one, so we’ll use React state to store that value, then apply it to our image.

We’ll use useState again and under our cldData state instance add:

const [filter, setFilter] = useState();

When that filter value is set, we’ll want to use it, so let’s add an if statement to our cldImage block to apply it if it’s available.

if ( cldImage ) {
  if ( filter ) {
    cldImage.effect(`e_art:${filter}`);
  }
  src = cldImage.toURL();
}

And finally, we need to trigger the state update.

Inside of our art filter list, we want to update the button element to include an onClick handler to set that filter value.

Update the button’s opening tag to:

<button className={styles.filterThumb} onClick={() => setFilter(filter)}>

Whenever someone clicks one of those filters, it will trigger that function, and update state.

Tip: Update the data-is-active-filter data attribute on the list item (li) and use that to style an active filter differently!

But now, whenever you click on one of the filters in the list, it will update it to that filter!

Now finally, we probably don’t want to show a generic thumbnail for each filter, we want to show a preview! So we can use the same URL gen SDK to add a preview for each filter.

We can do this by essentially copying the code we’re using to generate the main image, but dynamically configuring it for each thumb.

Replace the image inside the filter list with:

<img width="100" height="100" src={
  cloudinary.image(cldData?.public_id)
    .resize('w_200,h_200')
    .effect(`e_art:${filter}`)
    .toURL()
} alt={filter} />

Here we’re using the same image method to create our base URL and apply the effect. We’re also resizing the image to avoid serving an image larger than we need.

Note: the formatting in the code snippet above probably isn’t the easiest to read or to manage, but it’s probably the easiest way to show what’s happening in the snippet without too much code. Feel free to update it to your liking, including setting the image URL as a variable before the return statement!

But now we’ll see all of our images with previews!

Follow along with the commit!

Step 6: Adding masks on faces and overlay effects to an image

As another effect, we can take advantage of both the ability to overlay images on other images and Cloudinary’s face detection to add masks to people’s faces.

To start, we need an image uploaded to our Cloudinary account that we can use as an overlay.

I’m going to use a Darth Vader mask that I cut out from a Google search.

Tip: make sure you keep Copyright laws in mind when using this type of thing for your projects 🙂

After uploading your image to Cloudinary, you’ll want to take note of the public ID, as we’ll be using that in our overlay code.

To make this work, we’re going to use code very similar to our filters in step 5, so similar, we’re going to copy and paste most of it!

Before we do though, we need a list of our overlays that we’ll apply, much like the art filters list.

At the top of the file, next to the art filters list, add:

const OVERLAYS = [
  '<Your vader-helmet Public ID>'
];

Make sure to replace the value with your public ID and feel free to add more if you’ve uploaded more images!

Next we want to store the selected overlay, so we’ll use state like we did with our filters.

const [overlay, setOverlay] = useState();

Then, we’re going to loop through each overlay like our filters, so we’re going to copy our filter list block just like it is and update it to overlays.

Add the following above or below the filters list depending on the order you prefer.

<h2>Overlays</h2>
<ul className={styles.filters}>
  {OVERLAYS.map(overlay => {
    return (
      <li key={overlay} data-is-active-filter={false}>
        <button className={styles.filterThumb} onClick={() => setOverlay(overlay)}>
          <img width="100" height="100" src={
            cloudinary.image(cldData?.public_id)
              .resize('w_200,h_200')
              .toURL()
          } alt={overlay} />
          <span>{ overlay }</span>
        </button>
      </li>
    )
  })}
</ul>

We’ll see our overlays list load, but you’ll notice we’re not including any effects in there, let’s see how this is going to work.

To add our overlays, we’re going to use the addTransformation method instead of the effect method.

Particularly, we’re going to add a new layer, where we’ll then adjust that layer to where we want it to go.

In our example, we’re going to be overlaying a mask, so we want that layer to be positioned on someone’s face.

So let’s add our transformation to both our overlays list and our image if selected.

First, if the overlay is selected, let’s apply it to our main image:

if ( cldImage ) {
  if ( overlay ) {
    cldImage.addTransformation(`l_${overlay}/fl_layer_apply,fl_relative,g_faces,h_1.2,y_-0.05`)
  }
  if ( filter ) {
    cldImage.effect(`e_art:${filter}`);
  }
  src = cldImage.toURL();
}

Then let’s also add it to our overlays preview list:

<img width="100" height="100" src={
  cloudinary.image(cldData?.public_id)
    .resize('w_200,h_200')
    .addTransformation(`l_${overlay}/fl_layer_apply,fl_relative,g_faces,h_1.2,y_-0.05`)
    .toURL()
} alt={overlay} />

But now if we take a photo and select our Vader overlay, we should now see the mask both in the preview and on our main photo!

Follow along with the commit!

What else can we do?

The cool thing is we have a ton of capabilities we can explore with Cloudinary, even custom ones, to add new and fun effects to our photos.

Use AI to add overlays based on what’s in the photo

Whenever we upload our images to Cloudinary, we can use Google’s Vision AI Cloudinary add-on to automatically add tags to our images to see what’s detected inside.

Once we have that information, we can dynamically add things on top of our images, such as if we see something space-related, we can add rocket ships or stars.

Learn how to add tags to your images with How to Automatically Tag & Categorize Images Using AI with Google Vision & Cloudinary.

Automatically crop webcam images using face detection

We used face detection to add an overlay, but we can also use face detection to automatically crop and resize the image to center it in the space.

Learn how with How to Create Thumbnail Images Using Face Detection with Cloudinary.

Set up social sharing with Twitter auth

Once someone snags a photo, how about setting up the ability for someone to log in with Twitter and sharing it out?

Using NextAuth.js, we can easily set up oAuth with a ton of different providers.

Learn how to set up Twitter login with How to Authenticate Next.js Apps with Twitter & NextAuth.js.