The assistant follows a clean, modular architecture divided into seven core layers:

1. User Input Layer

Users interact through either text typing or voice input (using speech_recognition). This dual-mode interface ensures accessibility—type when you’re in a meeting, speak when your hands are busy.

2. Web Interface (Flask Frontend)

A responsive HTML/CSS/JS frontend serves as the control center. Built with Flask templates, it provides a chat-like interface (as seen in your NovaChat UI screenshots) where users can type commands, view responses, and even trigger camera feeds.

3. API Layer

The /chat endpoint (Flask backend) acts as the central router. Every user input—whether “What is generative AI?” or “open calculator”—hits this endpoint. The API then delegates to either the rule-based engine or the LLM.

4. Request Processing Engine (Rule-Based)

This is where the magic of system automation happens. The engine checks incoming text against predefined command patterns:

If a command matches, the assistant executes it locally—opening apps via webbrowser or os.system, or fetching time via datetime. No LLM inference is needed, meaning near-instant execution.

5. AI Engine (Local LLM with Ollama)

For anything the rule engine doesn’t recognize, the system falls back to the Mista model running via Ollama. This 7B-parameter LLM handles:

• Explanatory questions (“Explain generative AI in simple terms”)

• Creative writing

• General conversation

Because it’s local, inference times typically range from 2 to 5 seconds on a modern CPU (or faster with GPU acceleration). The response is then streamed back to the UI.

6. Response Generation & Output Layer

The final response is delivered in three ways:

• JSON payload to the frontend for display

• Text-to-speech audio via asynchronous queue

• Local execution (e.g., opening an app)

7. Speech System (Asynchronous Queue)

This is a critical architectural win. Instead of blocking the main thread with TTS, the assistant uses:

Speech_queue = queue. Queue()

speech_worker() # Runs in a separate thread

engine = pyttsx3.init()

All speak_text() calls simply push text into a queue. A background thread continuously processes this queue using pyttsx3, converting text to speech and playing audio. This means you can ask a question, receive a spoken answer, and type your next command simultaneously—no UI freezing.

Multi-Threaded Background Services

The assistant runs two key parallel threads that never interfere with the main interaction loop:

Camera Module (OpenCV)

When a user says “start camera,” a separate thread initializes OpenCV (cv2.VideoCapture). The video feed displays in a popup window while the assistant remains fully interactive. Commands like “stop camera” cleanly release the thread and free system resources. This is far superior to blocking implementations that would freeze the UI.

Speech Worker Thread

As mentioned, pyttsx3 runs in its thread. A while True loop waits for queue items, speaks them, then marks the task as done. When the application shuts down, a None sentinel gracefully terminates the thread.

Real-World Workflow: Step by Step

Let’s trace a complete user interaction:

1. User types, “What is the current time?”

2. Flask endpoint receives the request.

3. The rule engine matches the phrase to the utility module.

4. System fetches datetime.now().strftime(“%I:%M %p”).

5. Response: “The current time is 05:38 PM” is sent as JSON.

6. Simultaneously, the same text is pushed to speech_queue.

7. The speech worker speaks the time while the UI updates.

8. Total execution time: ~200 ms (no LLM involved).

Now, a non-command query:

1. User types: “Explain generative AI in simple terms.”

2. The rule engine finds no match.

3. Input forwarded to Ollama (Mista model).

4. LLM generates a human-like explanation.

5. Response returned, spoken, and displayed.