My Projects
Current and Previous Projects Descriptions
Massachusetts Institute of Technology (MIT) Media Labs (Ongoing)
- Developing methods to calibrate clinical Agent-Based Models(ABMs) directly from biopsies to have a mean accuracy of 77% under the Spatial Agreement Measure(SAM) Metric, minimizing the number of biopsy samples taken.
- Designing a novel multi-modal calibrated ABM pipeline to apply gradient-based ABMs to simulate tumour-immune cell interactions. (for Cytotoxic CD8+ T Cells in multiple carcinomas and melanoma cases)
Report
Hugging Face X Health (Ongoing)
- Developing novel models derived from the cumulative performance and extrapolation of Segment Anything Model (SAM), Medical SAM (Med-SAM), Fast-SAM.
- The results, when observed in Modalities such as Pathology, X-Ray, CT, and Ultrasound, gave an average improvement of 0.48 in mean Intersection of Union (mIOU) and 0.42 in Dice Score Coefficient (DSC).
Report
University of Utah
Biomedical Image Segmentation and Domain Adaptation
- The hypothesis revolves around the fact that the models like U-Net get biased when trained on a specific dataset like CRAG. Then it loses its accuracy when tested on a similar dataset like GLAS. Also true for other combinations of binary and multi-class segmentation datasets.
- Fine-tuned U-Net, DeepLabV3 model on Dataset like GlaS from MICCAI (2015), CRAG, CPM15 to observe domain dependency of models on the dataset, created a pipeline to improve Image masks mean Intersection of Union (mIOU) and Dice Score Coefficient (DSC).
Report
Domain-specific Question Answering chatbot
Project on problem statement given by DevRev.ai
We develop pipelines to retrieve a knowledge base article from the database based on the query and answer the query using the retrieved passage. We optimize the pipeline for performance, latency, and resource usage. Developed question-answering pipeline using techniques like model distillation, sparsification, pruning, and fine- tuning the DebertaV3-Base model to decrease inference time and have a minimum loss in accuracy.
Github Report
Captcha Breaker
Project by C&A Club, IITG
Deployed a Computer Vision program using Streamlit library that recognizes
Text-based Captcha images and converts them into writable text.
Developed the pipeline using Pytorch involving the RCNN model, giving
the CTC Loss as 0.03.
Re-colorisation of monochrome images using conditional GANs
Project by Coding Club, IITG
Trained a conditional Generative Adversarial Networks model (Discriminator and Generator) based on U-Net block with Resnet18 backbone and devised Image Processing strategies for colorization of monochrome images. Deployed a web app using Streamlit library on HuggingFace for the fine-tuned model over the COCO dataset.
Github
Cover Generation using OpenAI tools
Project by IITG.ai Club, IITG
Developed a multi-modal pipeline that converts audio/text input into
images using state-of-the-art OpenAI tools. Generated optimal transcripts for
the podcasts and songs with OpenAI Whisper to use in creating prompts.
Designed pipeline with Latent Diffusion Models (DALL-E) to generate
aesthetic cover images from created prompts using ChatGPT/GPT-2 models.
Super Resolution Photographic Mosaic
Project by Coding Club, IITG
Developed a Computer Vision pipeline that enhances the
images by super-resolution and image stitching.
Designed multi-model pipeline that consists of mainly Latent Diffusion Upscaler model
for super-resolution and
Image Stitcher for creating a panorama.