Browser Based Background Remover using ONNX Runtime Web

What is ONNX and ONNX Runtime Web?

ONNX stands for Open Neural Network Exchange, a standardized, open format for representing machine learning models.

Facebook and Microsoft introduced ONNX with the goal of solving interoperability issues across different ML models, making it easier to move models between frameworks and platforms.

ONNX Runtime Web provides a high-performance JavaScript inference engine that runs ONNX models directly within a web browser. 

It allows developers to run machine learning models on the browser, consequently enabling privacy-preserving and low-latency applications.

It uses WebAssembly (WASM) for its execution, enabling fast, portable, and secure inference directly in the browser.

Background Removal: A quick rundown

Background removal is a common computer vision task that separates the primary subject of an image—such as a human, product, or object—from its background.

Image segmentation models process the image by classifying each pixel as either foreground or background, and then remove all the background pixels from the image.

Traditionally, image background removal models runs on the server because they require high computational power, but this approach often leads to higher server costs and privacy concerns.

With the new approach where we utilize ONNX Runtime Web we can overcome these short Cummings providing a better interoperable solution.

Running these background removal models in the browser offers various advantages like privacy and inference cost. Thus, making it a crucial development.

Browser Based Background Remover: Our Approach

We have successfully built a browser-based background remover using ONNX Runtime Web as the inference engine.

As a result, the solution runs a pre-trained background removal model entirely inside the user’s browser using WebAssembly, eliminating the need for any server-side processing.

Now with this, we can seamlessly utilize browser resources to load the background remover model. In fact, this provides a more scalable solution that works across browser without extra dependency.

How our Background Remover Works

The background remover operates entirely inside the browser, following a simple yet efficient workflow that ensures performance, privacy, and flexibility.

1. Model Download and Caching:
   • When the web app loads for the first time, it downloads the model and stores it securely in IndexedDB for future use.
2. Inference Session:
   • The browser creates an inference session using ONNX Runtime Web and loads the model into that session.
3. Processing:
   • Once the model loads, the user selects an image and clicks the Process button to generate the masked image. This entire process happens locally within the browser.
   • After that, the system applies the mask to the original image, producing a version of the image without the background.
4. Custom Background:
   • We also provide functionality to user to choose custom background color or images. It provides the user with more control over the final image.

Advantages of Our Browser-Based Approach

• Complete Client-Side Execution: All processing happens inside user’s browser. This eliminates server dependency, reduces infrastructure costs, and ensures privacy.
• Improved Performance Through Caching: By storing the ONNX model in IndexedDB, we avoid repeated downloads. Once cached, the model loads quickly on subsequent visits.
• Cross-Platform Compatibility: The solution works seamlessly across different browsers on both desktop and mobile devices, without requiring any special hardware.
• Reduced server load: Client-side inference eliminates server resource usage, which lowers infrastructure costs.
• Scalable by default: Client-side execution automatically scales with the number of users, without requiring additional backend scaling.
• Easy integration: Developers can integrate it into existing web applications using JavaScript without complex setup or backend configuration.

Conclusion

By running the background removal model directly in the browser, we eliminated server-side processing and kept all user data on the client, creating a fast and privacy-focused workflow.

The initial loading time was reduced by storing the model in IndexedDB so it does not need to be downloaded again.

Moreover, We reduced the initial loading time by storing the model in IndexedDB, which prevents it from being downloaded again on every visit.

Although ONNX Runtime Web has some limitations compared to native runtimes, these issues were successfully managed with the right design choices.

As a result, we built a fast and privacy-friendly background removal tool that runs directly in the browser.

It clearly demonstrates that ONNX Runtime Web can be used effectively for real-world applications.