Monthly Devlog: Transcription & Diarization System Enhancements

Overview

This month, my primary focus was on refining the transcription and diarization pipeline to improve accuracy, performance, and user experience across platforms—especially addressing Mac-specific challenges. Key themes included centralizing styling for maintainability, optimizing subprocess management for transcription workflows, and enhancing offline capabilities by bundling models. Balancing app size, stability, and UI clarity were recurring considerations throughout.

Highlights & Progress

Codebase & Styling Consolidation

• Centralized all app styling into a single theme class, unifying color schemes, fonts, and styles for easier maintenance.
• Merged two similar manual transcription classes to simplify the codebase and reduce redundancy.
• Partitioned pro-user toggles and moved diarization speaker identification rules into the AI engine panel to streamline the UI and improve user clarity.

Transcription and Diarization Pipeline

• Resolved dependency conflicts by uninstalling the monolithic Hugging Face package and installing individual modules, stabilizing the transcription pipeline.
• Integrated OpenAI Whisper, Faster Whisper, and diarization as subprocesses that trigger automatically upon file detection, improving workflow automation.
• Upgraded the GPU torch package to CUDA 12.9, enabling GPU-accelerated transcription and diarization, significantly boosting performance on supported hardware.
• Shifted speaker assignment to occur after summarization based on LLM confidence scores—running diarization labels once the summary pass has enough context to judge speaker identity reliably.
• Refactored transcript chunking to break on logical boundaries such as sentences or paragraphs rather than arbitrary cuts, improving readability and coherence.

Platform-Specific Improvements (Mac Focus)

• Replaced OpenAI Whisper with Faster Whisper on Mac to address compatibility and performance issues related to weaker Intel hardware and lack of Metal support.
• Bundled the diarization model within the app for offline use, increasing app size by approximately 300MB but enabling functionality without internet connectivity.
• Developed a script to check and start Ollama on Mac to assist with hardware acceleration workflows.
• Despite ongoing Mac stability challenges, decided to keep hardware-accelerated transcription accessible to all users.
• Prepared a small beta testing group (~5 testers) for both Mac and Windows platforms to gather targeted feedback.

UI & Animation Enhancements

• Modified the animation engine to accept live app state inputs, reducing lag and improving responsiveness.
• Refreshed confirmation screens with looping blue circle animations and button depress effects to enhance user feedback.
• Added fallback URLs for hyperlinks and improved mode builder persistence to increase robustness.
• Addressed medium-priority bugs related to responsiveness, save button feedback, real-time confirmation feedback, button interactions, and smoothing subprocess calls.

Reflections

• Centralizing styling and merging transcription classes significantly improved maintainability and reduced complexity, validating the decision to consolidate early.
• Automating subprocess triggers for transcription and diarization streamlined workflows but surfaced new challenges in managing subprocess stability and feedback responsiveness.
• Bundling the diarization model offline was a tradeoff: it improved offline usability but increased app size notably, raising questions about compression and packaging strategies.
• Mac hardware limitations, especially on Intel models lacking Metal support, remain a persistent hurdle, requiring creative workarounds like switching to Faster Whisper and scripting Ollama management.
• The shift to assigning speakers post-summarization based on LLM confidence is promising but requires further validation to confirm diarization accuracy with multiple speakers.
• UI animation improvements enhanced user experience, but smoothing subprocess interactions and resolving related bugs will be critical to polish the final product.
• Preparing a small beta group is a prudent step to collect focused, actionable feedback before wider release.

Next Steps

• Investigate and implement a more effective model unloading mechanism to prevent VRAM bloat and app crashes during transcription and diarization.
• Explore compression or alternative packaging methods to reduce the app size impact from bundling the diarization model.
• Continue improving Mac hardware acceleration support, including refining Ollama management scripts and evaluating Metal-capable Mac models.
• Conduct thorough testing of diarized insights with multiple speakers to validate the new speaker assignment approach and diarization accuracy.
• Finalize UI feedback animations and smooth subprocess interactions while addressing outstanding medium-priority bugs to enhance overall stability and polish.

Keynotes

Decisions & Observations

• Decided to centralize all app styling into a single theme class and merge similar transcription classes to improve maintainability and reduce code complexity.
• Resolved dependency conflicts by replacing the monolithic Hugging Face package with individual modules to stabilize the transcription pipeline.
• Will investigate and implement improved model unloading mechanisms to prevent VRAM bloat and app crashes during transcription and diarization.

Issues & Next Steps

• Needs to resolve subprocess stability and feedback responsiveness issues caused by automated subprocess triggers.
• Needs to validate diarization accuracy and speaker assignment effectiveness with multiple speakers through thorough testing.
• Needs to address medium-priority UI bugs related to responsiveness, button feedback, and subprocess call smoothing to polish the product.
• Will explore compression or alternative packaging strategies to reduce app size impact from bundling the diarization model.
• Will continue refining Mac hardware acceleration support, including Ollama management scripts and evaluating Metal-capable Mac models.
• Will finalize UI feedback animations and smooth subprocess interactions to enhance stability and user experience.

Uncategorized Notes

• Integrated OpenAI Whisper, Faster Whisper, and diarization as automated subprocesses triggered on file detection to streamline workflows.
• Upgraded GPU torch package to CUDA 12.9 to enable GPU-accelerated transcription and diarization, boosting performance on supported hardware.
• Shifted speaker assignment to occur after summarization based on LLM confidence scores—applying labels once the LLM has enough context to judge speaker identity reliably.
• Replaced OpenAI Whisper with Faster Whisper on Mac to address compatibility and performance issues on Intel hardware lacking Metal support.
• Bundled the diarization model offline within the app to enable functionality without internet connectivity, accepting a 300MB app size increase.
• Developed a script to manage Ollama hardware acceleration workflows on Mac, while keeping hardware-accelerated transcription accessible despite stability challenges.
• Prepared a small beta testing group (~5 testers) on Mac and Windows to gather targeted feedback before wider release.

Generated with Nucleate by Watchlight Studio — distilled from full transcription.