ML/AI and IoT Explorations

Overview

Hospital Inpatient Experience to showcase Next Gen Surgery Experience

Description: This use case shows the Health client how Computer Vision can be used to ensure a safe surgery is conducted. In an Operating Theater, lots of instruments are passed around during a surgery. In certain instances, surgical instruments or medical supplies like sponges are misplaced or unaccounted for. In some serious cases surgeons leave a surgical instrument inside the body of a patient during surgery, known as Retained Surgical Instruments (RSI) incident. This can lead to serious health complications for patients and fines and loss of reputation for the hospital and surgeons

To solve for this we leverage a Computer Vision based solution that tracks the position of all medical instruments and supplies during a surgery and presents the output on an easy-to-read intelligent dashboard for tracking.

Two cameras are mounted, one on top of the surgical table and other focused on the OT tray. The Computer Vision algorithm tracks if an instrument or medical supply placed in the OT tray matches the supplies ordered for the surgery. It also tracks if the items are taken out of item tray. The algorithm keeps a running count of the instruments present on the surgical table. It also documents the last know location of items not in the item tray and provides timestamp with link to view the video during timestamp. The algorithm can also alert if there are any discrepancies such as placing an unordered surgical item in OT tray

Overview

AI Avatar creation using Comfy UI workflow and Caricature artwork

Workflow: Upscaling is done using Ultimate SD Upscale, and InstaID is used primarily for recognition.

Other Images

Workflow

Overview

Although AnimateDiff can provide modeling of animation streams, the differences in the images produced by Stable Diffusion still cause a lot of flickering and incoherence. As far as the current tools are concerned, IPAdapter with ControlNet OpenPose is the best solution to compensate for this problem.

Workflow: AnimateDiff + IPAdapter

Input Images - 1

Final Videos (1)

Input Images - 2

Final Videos (2)

Overview

Leveraging the power of TouchDesigner and Python, I am currently investigating the potential of AI in generating dynamic graphics in real time, and integrating them with motion cameras and electronic devices for interactivity.

Final Videos