Rishubh Parihar

I am a fourth year Ph.D. student at Indian Insitute of Science, Bangalore and associated with Video Analytics Lab . I am advised by Prof. Venkatesh Babu from Department of Computational and Data Sciences. Previously, I worked at ShareChat where I developed and deployed large scale machine learning algorithms for content moderation for social media platforms ShareChat and Moj. Even before that, I worked on developing computer vision solutions for smartphones at Samsung Research Bangalore. I graduated with Bachelor of Technology in Mathematics and Computing in 2018 from Indian Insitute of Technology Delhi.

My research focuses on developing approaches to enhance controllability and designing new interfaces to interact with large-scale image generative models.

Email  /  CV  /  Google Scholar  /  Linkedin  /  Github

Existing attribute editing methods treat semantic attributes as binary, resulting in a single edit per attribute. However, attributes such as eyeglasses, smiles, or hairstyles exhibit a vast range of diversity. In this work, we formulate the task of diverse attribute editing by modeling the multidimensional nature of attribute edits. We capitalize on disentangled latent spaces of pretrained GANs and train a Denoising Diffusion Probabilistic Model (DDPM) to learn the latent distribution for diverse edits. This leads to latent subspaces that enable diverse attribute editing. Applying diffusion in the highly compressed latent space allows us to model rich distributions of edits within limited computational resources

Existing NeRF approaches concentrate on modelling a single object or a single level of a scene. However, in the real world, we may capture a scene at multiple levels, resulting in a layered capture. For example, tourists usually capture a monument’s exterior structure before capturing the inner structure. We propose Strata-NeRF, a single neural radiance field that implicitly captures a scene with multiple levels. Strata-NeRF achieves this by conditioning the NeRFs on Vector Quantized (VQ) latent representations which allow sudden changes in scene structure.

In this work, we propose a novel method to decompose motion in videos by using a pretrained image GAN model. We discover disentangled motion subspaces in the latent space of widely used style-based GAN models that are semantically meaningful and control a single explainable motion component. The proposed method uses only a few (≈10) ground truth video sequences to obtain such subspaces.

In this work we proposed a Few-shot Latent based Attribute Manipulation and Editing (FLAME) method, a simple yet effective framework to perform highly controlled image editing by latent space manipulation. FLAME can generate highly realistic attribute edits and enables us to generate diverse attribute styles such as hair-styles, trained with only a few set of image pairs.

Proposed a hierarchical regularizer during the training of StyleGAN models to induce smoothness properties in the W/W+ latent spaces. The regularizer is implemented by guiding the intermediate features of the StyleGAN with the features from the pretrained feature extractors.

In this paper, we propose a novel approach for understanding object motions via motion type classification. The proposed motion type classifier predicts a motion type for the video based on the trajectories of the objects present. Our classifier assigns a motion type for the given video from the following five primitive motion classes: linear, projectile, oscillatory, local and random. We demonstrate that the representations learned from the motion type classification generalizes well for the challenging downstream task of video retrieval. Further, we proposed a recommendation system for video playback style based on the motion type classifier predictions.

We propose a real-time face softening approach that smooths blemishes in the facial skin region, followed by a wavelet band manipulation to restore the underlying skin texture, which produces a highly appealing ‘beautified’ face that retains its natural appearance.

Given a source and a target image transferring makeup from the source image to the target image. The transferred makeup should blend in the scene to provide natural look. To this end, we have developed a complete framework which firstly relights the subject image to match the illumination of the target image. We have generated 3D face models from single image and used them for realistic relighting. Following that layer wise decomposition is done for relit source and target image and blending is done within corresponding layers to transfer makeup. Finally we have additional modules in our framework to add facial accessories. As we have generated 3D models of the source and target faces we were able to add accessories directly on 3D models which resulted in natural looking output.

We empirically validate a generative model - PixelTrans- former, which infers distribution of the spatial signal given a sparse set of input samples ex. image from the few ob- served pixels. We tested the method under two scenarios; when sampler polls from 1.) a noisy representation and 2.) a low-frequency representation of the underlying spatial signal. Also, we evaluated the model for the different number of encoder and decoder layers. We use the Cifar10[1] dataset for all our experiments.

Implemented a climbing heuristic to render climbers on required objects in a scene to enhance content creation for computer graphics applications. Built a graph which is a minimal abstract representation of plant for production of leaves and branches. Traversed the nodes in graph with geometry with materials and textures that can be rendered

Modeled a frog to be represented as a hierarchical model with an articulated structure Created animation module by interpolating key frames with diffuse, specular and ambient components Made an interactive game with multiple frogs which run behind use controlled insects

Implemented recursive ray tracing to generate an image of virtually generated 3D model by tracing path of light through pixels Implemented global illumination model with reflection, refraction & shadows and local illumination model with diffuse, specular and ambient components