Merge pull request #74 from steveh8758/mow-branch

Add: windows version start_services

CONTRIBUTING.md

@@ -74,11 +74,13 @@ Push your changes to your fork and submit a pull request to the main branch of t
 
 Here are some high-priority tasks and areas where we need contributions:
 
-- Web Browsing: Implement autonomous web browsing capabilities for the assistant.
+- Web Browsing: Improve the autonomous web browsing capabilities for the assistant.
-- Multi-Agent System: Enhance the multi-agent functionality on the dev branch.
+- Graphical interface, a web graphical interface. (please ask first)
-- Memory & Recovery: Improve conversation compression.
+- Multi-Agent System: Enhance the planner agent for divide and conqueer for task (please ask first).
 - New Tools: Add support for additional programming languages or APIs.
-- Testing: Write comprehensive tests for existing and new features.
+- Multi-language support for Text to speech & speech to text (english, chinese, spanish first)
+- Testing: Write comprehensive tests for existing features.
+- Better readme image: make a better readme image (robot whale that use tools. Ghibli or anime style, inspiration could be https://sakana.ai/assets/ai-scientist/cover.jpeg)
 
 
 If you're unsure where to start, feel free to reach out by opening an issue or joining our community discussions.

Dockerfile

@@ -0,0 +1,26 @@
+# Use official Python 3.11 image as the base
+FROM python:3.11
+
+# Set working directory
+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    gcc \
+    g++ \
+    gfortran \
+    libportaudio2 \
+    portaudio19-dev \
+    ffmpeg \
+    libavcodec-dev \
+    libavformat-dev \
+    libavutil-dev \
+    chromium \
+    chromium-driver \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN pip cache purge
+
+COPY . .
+
+RUN BLIS_ARCH=generic pip install --no-cache-dir -r requirements.txt

README.md

@@ -4,12 +4,19 @@
 
 **A fully local alternative to Manus AI**, a voice-enabled AI assistant that codes, explores your filesystem, browse the web and correct it's mistakes all without sending a byte of data to the cloud. Built with reasoning models like DeepSeek R1, this autonomous agent runs entirely on your hardware, keeping your data private.
 
-[![Visit AgenticSeek](https://img.shields.io/static/v1?label=Website&message=AgenticSeek&color=blue&style=flat-square)](https://fosowl.github.io/agenticSeek.html) ![License](https://img.shields.io/badge/license-GPL--3.0-green) ![Discord](https://img.shields.io/badge/Discord-Join%20Us-7289DA?logo=discord&logoColor=white)
+[![Visit AgenticSeek](https://img.shields.io/static/v1?label=Website&message=AgenticSeek&color=blue&style=flat-square)](https://fosowl.github.io/agenticSeek.html) ![License](https://img.shields.io/badge/license-GPL--3.0-green) [![Discord](https://img.shields.io/badge/Discord-Join%20Us-7289DA?logo=discord&logoColor=white)](https://discord.gg/4Ub2D6Fj)
+
 > 🛠️ **Work in Progress** – Looking for contributors!
 
 ![alt text](./media/whale_readme.jpg)
 
+> *Do a web search to find tech startup in Japan working on cutting edge AI research*
 
+> *Make a snake game in Python*
+
+> *Scan my network with nmap, find out who is connected?*
+
+> *Hey can you find where is contract.pdf*?
 
 ## Features:
 
@@ -31,7 +38,7 @@
 
 - **Memory**: Remembers what’s useful, your preferences and past sessions conversation.
 
-- **Web Browsing**: Autonomous web navigation is underway.
+- **Web Browsing**: Autonomous web navigation.
 
 
 ### Searching the web with agenticSeek :
@@ -44,11 +51,16 @@
 
 ## **Installation**
 
-### 1️⃣ **Clone the repository**
+Make sure you have chrome driver, docker and python3.10 (or newer) installed.
+
+For issues related to chrome driver, see the **Chromedriver** section.
+
+### 1️⃣ **Clone the repository and setup**
 
 ```sh
 git clone https://github.com/Fosowl/agenticSeek.git
 cd agenticSeek
+mv .env.example .env
 ```
 
 ### 2️ **Create a virtual env**
@@ -84,10 +96,10 @@ python3 setup.py install
 
 Make sure you have [Ollama](https://ollama.com/) installed.
 
-Download the `deepseek-r1:7b` model from [DeepSeek](https://deepseek.com/models)
+Download the `deepseek-r1:14b` model from [DeepSeek](https://deepseek.com/models)
 
 ```sh
-ollama pull deepseek-r1:7b
+ollama pull deepseek-r1:14b
 ```
 
 ### 2️ **Run the Assistant (Ollama)**  
@@ -97,21 +109,22 @@ Start the ollama server
 ollama serve
 ```
 
-Change the config.ini file to set the provider_name to `ollama` and provider_model to `deepseek-r1:7b`
+Change the config.ini file to set the provider_name to `ollama` and provider_model to `deepseek-r1:14b`
 
-NOTE: `deepseek-r1:7b`is an example, use a bigger model if your hardware allow it.
+NOTE: `deepseek-r1:14b`is an example, use a bigger model if your hardware allow it.
 
 ```sh
 [MAIN]
 is_local = True
 provider_name = ollama
-provider_model = deepseek-r1:7b
+provider_model = deepseek-r1:14b
+provider_server_address = 127.0.0.1:11434
 ```
 
 start all services :
 
 ```sh
-./start_services.sh
+sudo ./start_services.sh
 ```
 
 Run the assistant:
@@ -120,6 +133,77 @@ Run the assistant:
 python3 main.py
 ```
 
+*See the **Usage** section if you don't understand how to use it*
+
+*See the **Known issues** section if you are having issues*
+
+*See the **Run with an API** section if your hardware can't run deepseek locally*
+
+---
+
+## Usage
+
+Warning: currently the system that choose the best AI agent routing system will work poorly with non-english text. This is because the agent routing currently use a model that was trained on english text. We are working hard to fix this. Please use english for now.
+
+
+Make sure the services are up and running with `./start_services.sh` and run the agenticSeek with `python3 main.py`
+
+```sh
+sudo ./start_services.sh
+python3 main.py
+```
+
+You will be prompted with `>>> `
+This indicate agenticSeek await you type for instructions.
+You can also use speech to text by setting `listen = True` in the config.
+
+Here are some example usage:
+
+### Coding/Bash
+
+> *Help me with matrix multiplication in Golang*
+
+> *Scan my network with nmap, find if any suspicious devices is connected*
+
+> *Make a snake game in python*
+
+### Web search
+
+> *Do a web search to find cool tech startup in Japan working on cutting edge AI research*
+
+> *Can you find on the internet who created agenticSeek?*
+
+> *Can you find on which website I can buy a rtx 4090 for cheap*
+
+### File system
+
+> *Hey can you find where is million_dollars_contract.pdf i lost it*
+
+> *Show me how much space I have left on my disk*
+
+> *Find and read the README.md and follow the install instruction*
+
+### Casual
+
+> *Tell me a joke*
+
+> *Where is flight ABC777 ? my mom is on that plane*
+
+> *what is the meaning of life ?*
+
+
+After you type your query, agenticSeek will allocate the best agent for the task.
+
+Because this is an early prototype, the agent routing system might not always allocate the right agent based on your query.
+
+Therefore, you should be very explicit in what you want and how the AI might proceed for example if you want it to conduct a web search, do not say:
+
+`Do you know some good countries for solo-travel?`
+
+Instead, ask:
+
+`Do a web search and find out which are the best country for solo-travel`
+
 ---
 
 ## **Run the LLM on your own server**  
@@ -148,7 +232,7 @@ Now on your personal computer:
 
 Clone the repository.
 
-Change the `config.ini` file to set the `provider_name` to `server` and `provider_model` to `deepseek-r1:7b`.
+Change the `config.ini` file to set the `provider_name` to `server` and `provider_model` to `deepseek-r1:14b`.
 Set the `provider_server_address` to the ip address of the machine that will run the model.
 
 ```sh
@@ -162,7 +246,7 @@ provider_server_address = x.x.x.x:5000
 Run the assistant:
 
 ```sh
-./start_services.sh
+sudo ./start_services.sh
 python3 main.py
 ```
 
@@ -183,12 +267,39 @@ provider_server_address = 127.0.0.1:5000 # can be set to anything, not used
 Run the assistant:
 
 ```sh
-./start_services.sh
+sudo ./start_services.sh
 python3 main.py
 ```
 
 ---
 
+
+
+## Speech to Text
+
+The speech-to-text functionality is disabled by default. To enable it, set the listen option to True in the config.ini file:
+
+```
+listen = True
+```
+
+When enabled, the speech-to-text feature listens for a trigger keyword, which is the agent's name, before it begins processing your input. You can customize the agent's name by updating the `agent_name` value in the *config.ini* file:
+
+```
+agent_name = Friday
+```
+
+For optimal recognition, we recommend using a common English name like "John" or "Emma" as the agent name
+
+Once you see the transcript start to appear, say the agent's name aloud to wake it up (e.g., "Friday").
+
+Speak your query clearly.
+
+End your request with a confirmation phrase to signal the system to proceed. Examples of confirmation phrases include:
+```
+"do it", "go ahead", "execute", "run", "start", "thanks", "would ya", "please", "okay?", "proceed", "continue", "go on", "do that", "go it", "do you understand?"
+```
+
 ## Providers
 
 The table below show the available providers:
@@ -212,13 +323,37 @@ provider_server_address = 127.0.0.1:5000
 ```
 `is_local`: should be True for any locally running LLM, otherwise False.
 
-`provider_name`: Select the provider to use by its name, see the provider list above.
+`provider_name`: Select the provider to use by it's name, see the provider list above.
 
 `provider_model`: Set the model to use by the agent.
 
 `provider_server_address`: can be set to anything if you are not using the server provider.
 
+# Known issues
+
+## Chromedriver Issues
+
+**Known error #1:** *chromedriver mismatch*
+
+`Exception: Failed to initialize browser: Message: session not created: This version of ChromeDriver only supports Chrome version 113
+Current browser version is 134.0.6998.89 with binary path`
+
+This happen if there is a mismatch between your browser and chromedriver version.
+
+You need to navigate to download the latest version:
+
+https://developer.chrome.com/docs/chromedriver/downloads
+
+If you're using Chrome version 115 or newer go to:
+
+https://googlechromelabs.github.io/chrome-for-testing/
+
+And download the chromedriver version matching your OS.
+
+![alt text](./media/chromedriver_readme.png)
+
 ## FAQ
+
 **Q: What hardware do I need?**  
 
 7B Model: GPU with 8GB VRAM.
@@ -233,10 +368,6 @@ Deepseek R1 excels at reasoning and tool use for its size. We think it’s a sol
 
 Ensure Ollama is running (`ollama serve`), your `config.ini` matches your provider, and dependencies are installed. If none work feel free to raise an issue.
 
-**Q: How to join the discord ?**
-
-Ask in the Community section for an invite.
-
 **Q: Can it really run 100% locally?**  
 
 Yes with Ollama or Server providers, all speech to text, LLM and text to speech model run locally. Non-local options (OpenAI or others API) are optional.
@@ -253,6 +384,8 @@ It's not but we prioritizes local execution and privacy over cloud based approac
 
 We’re looking for developers to improve AgenticSeek! Check out open issues or discussion.
 
+[![Star History Chart](https://api.star-history.com/svg?repos=Fosowl/agenticSeek&type=Date)](https://www.star-history.com/#Fosowl/agenticSeek&Date)
+
 ## Authors:
  > [Fosowl](https://github.com/Fosowl)
  > [steveh8758](https://github.com/steveh8758) 

config.ini

@@ -2,10 +2,10 @@
 is_local = True
 provider_name = ollama
 provider_model = deepseek-r1:14b
-provider_server_address = 127.0.0.1:5000
+provider_server_address = 127.0.0.1:11434
 agent_name = Friday
 recover_last_session = True
 save_session = False
 speak = True
 listen = False
-work_dir =  /Users/mlg/Documents/A-project/AI/Agents/agenticSeek/ai_workplace
+work_dir =  /Users/mlg/Documents/ai_workplace

main.py

@@ -41,10 +41,6 @@ def main():
                        name="File Agent",
                        prompt_path="prompts/file_agent.txt",
                        provider=provider),
-        PlannerAgent(model=config["MAIN"]["provider_model"],
-                       name="Planner",
-                       prompt_path="prompts/planner_agent.txt",
-                       provider=provider),
         BrowserAgent(model=config["MAIN"]["provider_model"],
                        name="Browser",
                        prompt_path="prompts/browser_agent.txt",

prompts/file_agent.txt

@@ -44,7 +44,7 @@ User: "I need to find the file config.txt and read its contents."
 
 Assistant: I’ll use file_finder to locate the file:
 
-```file_finder
+```file_finder:read
 config.txt
 ```
 

requirements.txt

@@ -1,32 +1,37 @@
-requests==2.31.0
+setuptools>=75.6.0
-openai==1.61.1
+requests>=2.31.0
-colorama==0.4.6
+numpy>=1.24.4
-python-dotenv==1.0.0
+colorama>=0.4.6
-playsound==1.3.0
+python-dotenv>=1.0.0
-soundfile==0.13.1
+playsound>=1.3.0
-transformers==4.48.3
+soundfile>=0.13.1
-torch==2.5.1
+transformers>=4.46.3
-ollama==0.4.7
+torch>=2.4.1
-scipy==1.15.1
+python-dotenv>=1.0.0
-kokoro==0.7.12
+ollama>=0.4.7
-flask==3.1.0
+scipy>=1.9.3
-soundfile==0.13.1
+kokoro>=0.7.12
-protobuf==3.20.3
+soundfile>=0.13.1
-termcolor==2.5.0
+protobuf>=3.20.3
-ipython==8.34.0
+termcolor>=2.4.0
-gliclass==0.1.8
+ipython>=8.13.0
-pyaudio==0.2.14
+pyaudio>=0.2.14
-librosa==0.10.2.post1
+librosa>=0.10.2.post1
-selenium==4.29.0
+selenium>=4.27.1
-markdownify==1.1.0
+markdownify>=1.1.0
+text2emotion>=0.0.5
+langid>=1.1.6
+chromedriver-autoinstaller>=0.6.4
 httpx>=0.27,<0.29
 anyio>=3.5.0,<5
 distro>=1.7.0,<2
 jiter>=0.4.0,<1
 sniffio
 tqdm>4
+# for api provider
+openai
 # if use chinese
 ordered_set
 pypinyin
 cn2an
-jieba
+jieba

scripts/linux_install.sh

@@ -5,6 +5,19 @@ echo "Starting installation for Linux..."
 # Update package list
 sudo apt-get update
 
+pip install --upgrade pip
+
+# install pyaudio
+pip install pyaudio
+# make sure essential tool are installed
+sudo apt install python3-dev python3-pip python3-wheel build-essential
+# install port audio
+sudo apt-get install portaudio19-dev python-pyaudio python3-pyaudio
+# install wheel
+pip install --upgrade pip setuptools wheel
+# install docker compose
+sudo apt install docker-compose
+
 # Install Python dependencies from requirements.txt
 pip3 install -r requirements.txt
 

searxng/.searxng.env

@@ -1 +0,0 @@
-SEARXNG_BASE_URL="http://127.0.0.1:8080"

searxng/docker-compose.yml

@@ -1,3 +1,4 @@
+version: '3'
 services:
   redis:
     container_name: redis

searxng/setup_searxng.sh

@@ -85,17 +85,6 @@ else
     exit 1
 fi
 
-# Stop containers
-echo "Stopping containers to apply security settings..."
-docker-compose down
-
-# Start containers again with secure settings
-echo "Deploying SearXNG with secure settings..."
-if ! docker-compose up -d; then
-    echo "Error: Failed to deploy SearXNG. Check logs with 'docker compose logs'."
-    exit 1
-fi
-
 # Display status and access instructions
 echo "SearXNG setup complete!"
 docker ps -a --filter "name=searxng" --filter "name=redis"

server/Dockerfile

@@ -0,0 +1,14 @@
+FROM ubuntu:20.04
+
+WORKDIR /app
+
+RUN apt-get update && \
+    apt-get install -y python3 python3-pip && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY requirements.txt .
+
+RUN pip3 install --no-cache-dir -r requirements.txt
+
+CMD ["python3", "--version"]

server/requirements.txt

@@ -0,0 +1,2 @@
+flask>=2.3.0
+ollama>=0.4.7

server/server.py

@@ -37,7 +37,10 @@ class GenerationState:
 
 state = GenerationState()
 
-def generate_response(history):  # Only takes history as an argument
+def generate_response_vllm(history):
+    pass
+
+def generate_response_ollama(history):  # Only takes history as an argument
     global state
     try:
         with state.lock:

setup.py

@@ -8,32 +8,34 @@ setup(
     version="0.1.0",
     author="Fosowl",
     author_email="mlg.fcu@gmail.com",
-    description="A Python project for agentic search and processing",
+    description="The open, local alternative to ManusAI",
     long_description=long_description,
     long_description_content_type="text/markdown",
     url="https://github.com/Fosowl/agenticSeek",
     packages=find_packages(),
     include_package_data=True,
     install_requires=[
-        "requests==2.31.0",
+        "requests>=2.31.0",
-        "openai==1.61.1",
+        "openai",
-        "colorama==0.4.6",
+        "colorama>=0.4.6",
-        "python-dotenv==1.0.0",
+        "python-dotenv>=1.0.0",
-        "playsound==1.3.0",
+        "playsound>=1.3.0",
-        "soundfile==0.13.1",
+        "soundfile>=0.13.1",
-        "transformers==4.48.3",
+        "transformers>=4.46.3",
-        "torch==2.5.1",
+        "torch>=2.4.1",
-        "ollama==0.4.7",
+        "ollama>=0.4.7",
-        "scipy==1.15.1",
+        "scipy>=1.9.3",
-        "kokoro==0.7.12",
+        "kokoro>=0.7.12",
-        "flask==3.1.0",
+        "flask>=3.1.0",
-        "protobuf==3.20.3",
+        "protobuf>=3.20.3",
-        "termcolor==2.5.0",
+        "termcolor>=2.5.0",
-        "gliclass==0.1.8",
+        "ipython>=8.34.0",
-        "ipython==8.34.0",
+        "librosa>=0.10.2.post1",
-        "librosa==0.10.2.post1",
+        "selenium>=4.29.0",
-        "selenium==4.29.0",
+        "markdownify>=1.1.0",
-        "markdownify==1.1.0",
+        "text2emotion>=0.0.5",
+        "python-dotenv>=1.0.0",
+        "langid>=1.1.6",
         "httpx>=0.27,<0.29",
         "anyio>=3.5.0,<5",
         "distro>=1.7.0,<2",
@@ -59,5 +61,5 @@ setup(
         "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
         "Operating System :: OS Independent",
     ],
-    python_requires=">=3.6",
+    python_requires=">=3.9",
 )

sources/agents/browser_agent.py

@@ -5,6 +5,7 @@ from sources.utility import pretty_print, animate_thinking
 from sources.agents.agent import Agent
 from sources.tools.searxSearch import searxSearch
 from sources.browser import Browser
+from datetime import date
 
 class BrowserAgent(Agent):
     def __init__(self, model, name, prompt_path, provider):
@@ -15,13 +16,16 @@ class BrowserAgent(Agent):
         self.tools = {
             "web_search": searxSearch(),
         }
-        self.role = "deep research and web search"
+        self.role = "Web Research"
         self.browser = Browser()
-        self.browser.go_to("https://github.com/")
         self.search_history = []
         self.navigable_links = []
         self.notes = []
+        self.date = self.get_today_date()
     
+    def get_today_date(self) -> str:
+        date_time = date.today()
+        return date_time.strftime("%B %d, %Y")
 
     def extract_links(self, search_result: str):
         pattern = r'(https?://\S+|www\.\S+)'
@@ -50,7 +54,7 @@ class BrowserAgent(Agent):
         {search_choice}
         Your goal is to find accurate and complete information to satisfy the user’s request.
         User request: {user_prompt}
-        To proceed, choose a relevant link from the search results. Announce your choice by saying: "I want to navigate to <link>."
+        To proceed, choose a relevant link from the search results. Announce your choice by saying: "I want to navigate to <link>"
         Do not explain your choice.
         """
     
@@ -58,31 +62,52 @@ class BrowserAgent(Agent):
         remaining_links = self.get_unvisited_links() 
         remaining_links_text = remaining_links if remaining_links is not None else "No links remaining, proceed with a new search." 
         return f"""
-        \nYou are currently browsing the web. Not the user, you are the browser.
+        You are a web browser.
-
+        You are currently on this webpage:
-        Page content:
         {page_text}
 
-        You can navigate to these links:
+        You can navigate to these navigation links:
         {remaining_links}
 
-        If no link seem appropriate, please say "GO_BACK".
+        Your task:
-        Remember, you seek the information the user want.
+        1. Decide if the current page answers the user’s query: {user_prompt}
-        The user query was : {user_prompt}
+          - If it does, take notes of the useful information, write down source, link or reference, then move to a new page.
-        You must choose a link (write it down) to navigate to, or go back.
+          - If it does and you are 100% certain that it provide a definive answer, say REQUEST_EXIT
-        For exemple you can say: i want to go to www.wikipedia.org/cats
+          - If it doesn’t, say: Error: This page does not answer the user’s query then go back or navigate to another link.
-        Always end with a sentence that summarize when useful information is found for exemple:
+        2. Navigate by either: 
-        Summary: According to https://karpathy.github.io/ LeCun net is the earliest real-world application of a neural net"
+          - Navigate to a navigation links (write the full URL, e.g., www.example.com/cats).
-        Do not say "according to this page", always write down the whole link.
+          - If no link seems helpful, say: GO_BACK.
-        If a website does not have usefull information say Error, for exemple:
+        
-        Error: This forum does not discus anything that can answer the user query
+        Recap of note taking:
-        Do not explain your choice, be short, concise.
+        If useful -> Note: [Briefly summarize the key information that answers the user’s query.]
+        Do not write "The page talk about ...", write your finding on the page and how they contribute to an answer.
+        If not useful -> Error: [Explain why the page doesn’t help.]
+        
+        Example 1 (useful page, no need of going futher):
+        Note: According to karpathy site (https://karpathy.github.io/) LeCun net is the earliest real-world application of a neural net"
+        No link seem useful to provide futher information. GO_BACK
+
+        Example 2 (not useful, but related link):
+        Error: This forum reddit.com/welcome does not discuss anything related to the user’s query.
+        There is a link that could lead to the information, I want to navigate to http://reddit.com/r/locallama
+
+        Example 3 (not useful, no related links):
+        Error: x.com does not discuss anything related to the user’s query and no navigation link are usefull
+        GO_BACK
+
+        Example 3 (query answer found):
+        Note: I found on github.com that agenticSeek is Fosowl.
+        Given this information, given this I should exit the web browser. REQUEST_EXIT
+
+        Current date: {self.date}
+        Remember, the user asked: {user_prompt}
+        Do not explain your choice.
         """
     
     def llm_decide(self, prompt):
         animate_thinking("Thinking...", color="status")
         self.memory.push('user', prompt)
-        answer, reasoning = self.llm_request(prompt)
+        answer, reasoning = self.llm_request()
         pretty_print("-"*100)
         pretty_print(answer, color="output")
         pretty_print("-"*100)
@@ -120,25 +145,49 @@ class BrowserAgent(Agent):
     def save_notes(self, text):
         lines = text.split('\n')
         for line in lines:
-            if "summary:" in line.lower():
+            if "note" in line.lower():
                 self.notes.append(line)
     
     def conclude_prompt(self, user_query):
-        search_note = '\n -'.join(self.notes)
+        annotated_notes = [f"{i+1}: {note.lower().replace('note:', '')}" for i, note in enumerate(self.notes)]
+        search_note = '\n'.join(annotated_notes)
+        print("AI research notes:\n", search_note)
         return f"""
-        Following a web search about:
+        Following a human request:
         {user_query}
-        Write a conclusion based on these notes:
+        A web AI made the following finding across different pages:
         {search_note}
+
+        Summarize the finding, and provide a conclusion that answer the request.
+        """
+    
+    def search_prompt(self, user_prompt):
+        return f"""
+        Current date: {self.date}
+        Make a efficient search engine query to help users with their request:
+        {user_prompt}
+        Example:
+        User: "search: hey jarvis i want you to login to my twitter and say hello everyone "
+        You: Twitter 
+
+        User: "I need info on the best laptops for AI this year."
+        You: "search: best laptops 2025 to run Machine Learning model, reviews"
+
+        User: "Search for recent news about space missions."
+        You: "search: Recent space missions news, {self.date}"
+
+        Do not explain, do not write anything beside the search query.
         """
 
     def process(self, user_prompt, speech_module) -> str:
         complete = False
 
+        animate_thinking(f"Thinking...", color="status")
+        self.memory.push('user', self.search_prompt(user_prompt))
+        ai_prompt, _ = self.llm_request()
         animate_thinking(f"Searching...", color="status")
-        search_result_raw = self.tools["web_search"].execute([user_prompt], False)
+        search_result_raw = self.tools["web_search"].execute([ai_prompt], False)
-        search_result = self.jsonify_search_results(search_result_raw)
+        search_result = self.jsonify_search_results(search_result_raw)[:7] # until futher improvement
-        search_result = search_result[:10] # until futher improvement
         prompt = self.make_newsearch_prompt(user_prompt, search_result)
         unvisited = [None]
         while not complete:
@@ -148,14 +197,14 @@ class BrowserAgent(Agent):
                 complete = True
                 break
             links = self.extract_links(answer)
+            if len(unvisited) == 0:
+                break
             if len(links) == 0 or "GO_BACK" in answer:
                 unvisited = self.select_unvisited(search_result)
                 prompt = self.make_newsearch_prompt(user_prompt, unvisited)
                 pretty_print(f"Going back to results. Still {len(unvisited)}", color="warning")
                 links = []
                 continue
-            if len(unvisited) == 0:
-                break
             animate_thinking(f"Navigating to {links[0]}", color="status")
             speech_module.speak(f"Navigating to {links[0]}")
             self.browser.go_to(links[0])
@@ -164,12 +213,12 @@ class BrowserAgent(Agent):
             self.navigable_links = self.browser.get_navigable()
             prompt = self.make_navigation_prompt(user_prompt, page_text)
 
-        speech_module.speak(answer)
         self.browser.close()
         prompt = self.conclude_prompt(user_prompt)
-        answer, reasoning = self.llm_request(prompt)
+        self.memory.push('user', prompt)
+        answer, reasoning = self.llm_request()
         pretty_print(answer, color="output")
         return answer, reasoning
 
 if __name__ == "__main__":
-    browser = Browser()
+    pass

sources/agents/casual_agent.py

@@ -18,7 +18,7 @@ class CasualAgent(Agent):
             "file_finder": FileFinder(),
             "bash": BashInterpreter()
         }
-        self.role = "casual talking"
+        self.role = "Chat and Conversation"
     
     def process(self, prompt, speech_module) -> str:
         complete = False

sources/agents/code_agent.py

@@ -20,7 +20,7 @@ class CoderAgent(Agent):
             "go": GoInterpreter(),
             "file_finder": FileFinder()
         }
-        self.role = "coding and programming"
+        self.role = "Code Assistance"
 
     def process(self, prompt, speech_module) -> str:
         answer = ""

sources/agents/file_agent.py

@@ -14,7 +14,7 @@ class FileAgent(Agent):
             "file_finder": FileFinder(),
             "bash": BashInterpreter()
         }
-        self.role = "files operations"
+        self.role = "find and read files"
     
     def process(self, prompt, speech_module) -> str:
         complete = False

sources/agents/planner_agent.py

@@ -3,7 +3,7 @@ from sources.utility import pretty_print, animate_thinking
 from sources.agents.agent import Agent
 from sources.agents.code_agent import CoderAgent
 from sources.agents.file_agent import FileAgent
-from sources.agents.casual_agent import CasualAgent
+from sources.agents.browser_agent import BrowserAgent
 from sources.tools.tools import Tools
 
 class PlannerAgent(Agent):
@@ -19,9 +19,9 @@ class PlannerAgent(Agent):
         self.agents = {
             "coder": CoderAgent(model, name, prompt_path, provider),
             "file": FileAgent(model, name, prompt_path, provider),
-            "web": CasualAgent(model, name, prompt_path, provider)
+            "web": BrowserAgent(model, name, prompt_path, provider)
         }
-        self.role = "complex programming tasks and web research"
+        self.role = "Research, setup and code"
         self.tag = "json"
 
     def parse_agent_tasks(self, text):

sources/browser.py

@@ -5,11 +5,17 @@ from selenium.webdriver.common.by import By
 from selenium.webdriver.support.ui import WebDriverWait
 from selenium.webdriver.support import expected_conditions as EC
 from selenium.common.exceptions import TimeoutException, WebDriverException
+from selenium.webdriver.chrome.options import Options
+import chromedriver_autoinstaller
 import time
+import os
+import shutil
 from bs4 import BeautifulSoup
 import markdownify
 import logging
 import sys
+import re
+from urllib.parse import urlparse
 
 class Browser:
     def __init__(self, headless=False, anticaptcha_install=False):
@@ -23,41 +29,92 @@ class Browser:
         self.anticaptcha = "https://chrome.google.com/webstore/detail/nopecha-captcha-solver/dknlfmjaanfblgfdfebhijalfmhmjjjo/related"
         try:
             chrome_options = Options()
+            chrome_path = self.get_chrome_path()
+            
+            if not chrome_path:
+                raise FileNotFoundError("Google Chrome not found. Please install it.")
+            chrome_options.binary_location = chrome_path
+            
             if headless:
                 chrome_options.add_argument("--headless")
             chrome_options.add_argument("--disable-gpu")
             chrome_options.add_argument("--no-sandbox")
             chrome_options.add_argument("--disable-dev-shm-usage")
-            self.driver = webdriver.Chrome(options=chrome_options)
+            chrome_options.add_argument("--autoplay-policy=user-gesture-required")
+            chrome_options.add_argument("--mute-audio")
+            chrome_options.add_argument("--disable-webgl")
+            chrome_options.add_argument("--disable-notifications")
+            security_prefs = {
+                "profile.default_content_setting_values.media_stream": 2,  # Block webcam/mic
+                "profile.default_content_setting_values.notifications": 2,  # Block notifications
+                "profile.default_content_setting_values.popups": 2,  # Block pop-ups
+                "profile.default_content_setting_values.geolocation": 2,  # Block geolocation
+                "download_restrictions": 3,  # Block all downloads
+                "safebrowsing.enabled": True,  # Enable safe browsing
+            }
+            chrome_options.add_experimental_option("prefs", security_prefs)
+            
+            chromedriver_path = shutil.which("chromedriver") # system installed driver.
+            
+            #If not found, try auto-installing the correct version
+            if not chromedriver_path:
+                chromedriver_path = chromedriver_autoinstaller.install()
+          
+            if not chromedriver_path:
+                raise FileNotFoundError("ChromeDriver not found. Please install it or add it to your PATH.")
+                
+            service = Service(chromedriver_path)
+            self.driver = webdriver.Chrome(service=service, options=chrome_options)
             self.wait = WebDriverWait(self.driver, 10)
             self.logger = logging.getLogger(__name__)
             self.logger.info("Browser initialized successfully")
         except Exception as e:
             raise Exception(f"Failed to initialize browser: {str(e)}")
-    
+            
+    @staticmethod
+    def get_chrome_path():
+        if sys.platform.startswith("win"):
+            paths = [
+                "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
+                "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
+                os.path.join(os.environ.get("LOCALAPPDATA", ""), "Google\\Chrome\\Application\\chrome.exe")  # User install
+            ]
+        elif sys.platform.startswith("darwin"):  # macOS
+            paths = ["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
+                     "/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"]
+        else:  # Linux
+            paths = ["/usr/bin/google-chrome", "/usr/bin/chromium-browser", "/usr/bin/chromium"]
+
+        for path in paths:
+            if os.path.exists(path) and os.access(path, os.X_OK):  # Check if executable
+                return path
+        return None
+
     def go_to(self, url):
         """Navigate to a specified URL."""
         try:
+            initial_handles = self.driver.window_handles
             self.driver.get(url)
-            time.sleep(2)  # Wait for page to load
+            time.sleep(1)
+            self.apply_web_countermeasures()
             self.logger.info(f"Navigated to: {url}")
             return True
         except WebDriverException as e:
             self.logger.error(f"Error navigating to {url}: {str(e)}")
             return False
-    
+
     def is_sentence(self, text):
         """Check if the text qualifies as a meaningful sentence or contains important error codes."""
         text = text.strip()
+
         error_codes = ["404", "403", "500", "502", "503"]
         if any(code in text for code in error_codes):
             return True
-        words = text.split()
+        words = re.findall(r'\w+', text, re.UNICODE)
         word_count = len(words)
-        has_punctuation = text.endswith(('.', '!', '?'))
+        has_punctuation = any(text.endswith(p) for p in ['.', '，', ',', '!', '?', '。', '！', '？', '।', '۔'])
         is_long_enough = word_count > 5
-        has_letters = any(word.isalpha() for word in words)
+        return (word_count >= 5 and (has_punctuation or is_long_enough))
-        return (word_count >= 5 and (has_punctuation or is_long_enough) and has_letters)
 
     def get_text(self):
         """Get page text and convert it to README (Markdown) format."""
@@ -72,15 +129,14 @@ class Browser:
             lines = (line.strip() for line in text.splitlines())
             chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
             text = "\n".join(chunk for chunk in chunks if chunk and self.is_sentence(chunk))
-            
+            #markdown_text = markdownify.markdownify(text, heading_style="ATX")
-            markdown_text = markdownify.markdownify(text, heading_style="ATX")
+            return "[Start of page]\n" + text + "\n[End of page]"
-            
-            return markdown_text
         except Exception as e:
             self.logger.error(f"Error getting text: {str(e)}")
             return None
     
     def clean_url(self, url):
+        """Clean URL to keep only the part needed for navigation to the page"""
         clean = url.split('#')[0]
         parts = clean.split('?', 1)
         base_url = parts[0]
@@ -95,6 +151,22 @@ class Browser:
             if essential_params:
                 return f"{base_url}?{'&'.join(essential_params)}"
         return base_url
+    
+    def is_link_valid(self, url):
+        """Check if a URL is a valid link (page, not related to icon or metadata)."""
+        if len(url) > 64:
+            return False
+        parsed_url = urlparse(url)
+        if not parsed_url.scheme or not parsed_url.netloc:
+            return False
+        if re.search(r'/\d+$', parsed_url.path):
+            return False
+        image_extensions = ['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp']
+        metadata_extensions = ['.ico', '.xml', '.json', '.rss', '.atom']
+        for ext in image_extensions + metadata_extensions:
+            if url.lower().endswith(ext):
+                return False
+        return True
 
     def get_navigable(self):
         """Get all navigable links on the current page."""
@@ -112,7 +184,7 @@ class Browser:
                     })
             
             self.logger.info(f"Found {len(links)} navigable links")
-            return [self.clean_url(link['url']) for link in links if link['is_displayed'] == True and len(link) < 256]
+            return [self.clean_url(link['url']) for link in links if (link['is_displayed'] == True and self.is_link_valid(link['url']))]
         except Exception as e:
             self.logger.error(f"Error getting navigable links: {str(e)}")
             return []
@@ -160,6 +232,130 @@ class Browser:
             self.logger.error(f"Error taking screenshot: {str(e)}")
             return False
 
+#######################
+#      WEB SECURITY   #
+#######################
+
+    def apply_web_countermeasures(self):
+        """
+        Apply security measures to block any website malicious execution, privacy violation etc..
+        """
+        self.inject_safety_script()
+        self.neutralize_event_listeners()
+        self.monitor_and_reset_css()
+        self.block_clipboard_access()
+        self.limit_intervals_and_timeouts()
+        self.block_external_requests()
+        self.monitor_and_close_popups()
+
+    def inject_safety_script(self):
+        script = """
+        // Block hardware access by removing or disabling APIs
+        Object.defineProperty(navigator, 'serial', { get: () => undefined });
+        Object.defineProperty(navigator, 'hid', { get: () => undefined });
+        Object.defineProperty(navigator, 'bluetooth', { get: () => undefined });
+        // Block media playback
+        HTMLMediaElement.prototype.play = function() {
+            this.pause(); // Immediately pause if play is called
+            return Promise.reject('Blocked by script');
+        };
+        // Block fullscreen requests
+        Element.prototype.requestFullscreen = function() {
+            console.log('Blocked fullscreen request');
+            return Promise.reject('Blocked by script');
+        };
+        // Block pointer lock
+        Element.prototype.requestPointerLock = function() {
+            console.log('Blocked pointer lock');
+        };
+        // Block iframe creation (optional, since browser already blocks these)
+        const originalCreateElement = document.createElement;
+        document.createElement = function(tagName) {
+            if (tagName.toLowerCase() === 'iframe') {
+                console.log('Blocked iframe creation');
+                return null;
+            }
+            return originalCreateElement.apply(this, arguments);
+        };
+        // Block annoying dialogs
+        window.alert = function() {};
+        window.confirm = function() { return false; };
+        window.prompt = function() { return null; };
+        """
+        self.driver.execute_script(script)
+
+    def neutralize_event_listeners(self):
+        script = """
+        const originalAddEventListener = EventTarget.prototype.addEventListener;
+        EventTarget.prototype.addEventListener = function(type, listener, options) {
+            if (['mousedown', 'mouseup', 'click', 'touchstart', 'keydown', 'keyup', 'keypress'].includes(type)) {
+                console.log(`Blocked adding listener for ${type}`);
+                return;
+            }
+            originalAddEventListener.apply(this, arguments);
+        };
+        """
+        self.driver.execute_script(script)
+
+    def monitor_and_reset_css(self):
+        script = """
+        const observer = new MutationObserver((mutations) => {
+            mutations.forEach((mutation) => {
+                if (mutation.type === 'attributes' && mutation.attributeName === 'style') {
+                    const html = document.querySelector('html');
+                    if (html.style.cursor === 'none') {
+                        html.style.cursor = 'auto';
+                    }
+                }
+            });
+        });
+        observer.observe(document.querySelector('html'), { attributes: true });
+        """
+        self.driver.execute_script(script)
+
+    def block_clipboard_access(self):
+        script = """
+        navigator.clipboard.readText = function() {
+            console.log('Blocked clipboard read');
+            return Promise.reject('Blocked');
+        };
+        navigator.clipboard.writeText = function() {
+            console.log('Blocked clipboard write');
+            return Promise.resolve();
+        };
+        """
+        self.driver.execute_script(script)
+
+    def limit_intervals_and_timeouts(self):
+        script = """
+        const originalSetInterval = window.setInterval;
+        window.setInterval = function(callback, delay) {
+            if (typeof callback === 'function' && callback.toString().includes('alert')) {
+                console.log('Blocked suspicious interval');
+                return;
+            }
+            return originalSetInterval.apply(this, arguments);
+        };
+        """
+        self.driver.execute_script(script)
+
+    def monitor_and_close_popups(self):
+        initial_handles = self.driver.window_handles
+        for handle in self.driver.window_handles:
+            if handle not in initial_handles:
+                self.driver.switch_to.window(handle)
+                self.driver.close()
+        self.driver.switch_to.window(self.driver.window_handles[0])
+
+    def block_external_requests(self):
+        script = """
+        window.fetch = function() {
+            console.log('Blocked fetch request');
+            return Promise.reject('Blocked');
+        };
+        """
+        self.driver.execute_script(script)
+
     def close(self):
         """Close the browser."""
         try:
@@ -178,11 +374,16 @@ if __name__ == "__main__":
     browser = Browser(headless=False)
     
     try:
-        browser.go_to("https://karpathy.github.io/")
+        # stress test
+        browser.go_to("https://www.bbc.com/news")
         text = browser.get_text()
         print("Page Text in Markdown:")
         print(text)
         links = browser.get_navigable()
         print("\nNavigable Links:", links)
+        print("WARNING SECURITY STRESS TEST WILL BE RUN IN 20s")
+        time.sleep(20)
+        browser.go_to("https://theannoyingsite.com/")
+        time.sleep(15)
     finally:
-        browser.close()
+        browser.close()

sources/interaction.py

@@ -27,7 +27,7 @@ class Interaction:
             self.transcriber = AudioTranscriber(self.ai_name, verbose=False)
             self.recorder = AudioRecorder()
         if tts_enabled:
-            self.speech.speak("Hello Sir, we are online and ready. What can I do for you ?")
+            self.speech.speak("Hello, we are online and ready. What can I do for you ?")
         if recover_last_session:
             self.recover_last_session()
     
@@ -58,7 +58,7 @@ class Interaction:
         buffer = ""
 
         PROMPT = "\033[1;35m➤➤➤ \033[0m"
-        while buffer == "" or buffer.isascii() == False:
+        while not buffer:
             try:
                 buffer = input(PROMPT)
             except EOFError:
@@ -98,10 +98,11 @@ class Interaction:
         agent = self.router.select_agent(self.last_query)
         if agent is None:
             return
-        if self.current_agent != agent:
+        if self.current_agent != agent and self.last_answer is not None:
-            self.current_agent = agent
+            ## get history from previous agent, good ?
-            # get history from previous agent, good ?
             self.current_agent.memory.push('user', self.last_query)
+            self.current_agent.memory.push('assistant', self.last_answer)
+        self.current_agent = agent
         self.last_answer, _ = agent.process(self.last_query, self.speech)
     
     def show_answer(self) -> None:

sources/language.py

@@ -0,0 +1,84 @@
+import langid
+import re
+import nltk
+from nltk.sentiment.vader import SentimentIntensityAnalyzer
+
+class LanguageUtility:
+    """LanguageUtility for language, or emotion identification"""
+    def __init__(self):
+        try:
+            nltk.data.find('vader_lexicon')
+        except LookupError:
+            nltk.download('vader_lexicon')
+        self.sid = SentimentIntensityAnalyzer()
+    
+    def detect_language(self, text: str) -> str:
+        """
+        Detect the language of the given text using langdetect
+        Args:
+            text: string to analyze
+        Returns: ISO639-1 language code
+        """
+        langid.set_languages(['fr', 'en', 'zh', 'es'])  # ISO 639-1 codes
+        lang, score = langid.classify(text)
+        return lang
+
+    def detect_emotion(self, text: str) -> str:
+        """
+        Detect the dominant emotion in the given text
+        Args:
+            text: string to analyze
+        Returns: string of the dominant emotion
+        """
+        try:
+            scores = self.sid.polarity_scores(text)
+            emotions = {
+                'Happy': max(scores['pos'], 0),
+                'Angry': 0,
+                'Sad': max(scores['neg'], 0),
+                'Fear': 0,
+                'Surprise': 0
+            }
+            if scores['compound'] < -0.5:
+                emotions['Angry'] = abs(scores['compound']) * 0.5
+                emotions['Fear'] = abs(scores['compound']) * 0.5
+            elif scores['compound'] > 0.5:
+                emotions['Happy'] = scores['compound']
+                emotions['Surprise'] = scores['compound'] * 0.5
+            dominant_emotion = max(emotions, key=emotions.get)
+            if emotions[dominant_emotion] == 0:
+                return 'Neutral'
+            return dominant_emotion
+        except Exception as e:
+            raise e
+    
+    def analyze(self, text):
+        """
+        Combined analysis of language and emotion
+        Args:
+            text: string to analyze
+        Returns: dictionary with language and emotion results
+        """
+        try:
+            language = self.detect_language(text)
+            emotions = self.detect_emotion(text)
+            return {
+                "language": language,
+                "emotions": emotions
+            }
+        except Exception as e:
+            raise e
+
+if __name__ == "__main__":
+    detector = LanguageUtility()
+    
+    test_texts = [
+        "I am so happy today!",
+        "Qué tristeza siento ahora",
+        "我不要去巴黎",
+        "La vie c'est cool"
+    ]
+    for text in test_texts:
+        print(f"\nAnalyzing: {text}")
+        result = detector.analyze(text)
+        print(result)

sources/llm_provider.py

@@ -12,6 +12,8 @@ from huggingface_hub import InferenceClient
 import os
 import httpx
 
+from sources.utility import pretty_print, animate_thinking
+
 class Provider:
     def __init__(self, provider_name, model, server_address = "127.0.0.1:5000"):
         self.provider_name = provider_name.lower()
@@ -21,17 +23,18 @@ class Provider:
             "ollama": self.ollama_fn,
             "server": self.server_fn,
             "openai": self.openai_fn,
-            "huggingface": self.huggingface_fn
+            "huggingface": self.huggingface_fn,
+            "deepseek-api": self.deepseek_fn
         }
         self.api_key = None
-        self.unsafe_providers = ["openai"]
+        self.unsafe_providers = ["openai", "deepseek-api"]
         if self.provider_name not in self.available_providers:
             raise ValueError(f"Unknown provider: {provider_name}")
         if self.provider_name in self.unsafe_providers:
-            print("Warning: you are using an API provider. You data will be sent to the cloud.")
+            pretty_print("Warning: you are using an API provider. You data will be sent to the cloud.", color="warning")
-            self.get_api_key(self.provider_name)
+            self.api_key = self.get_api_key(self.provider_name)
-        elif self.server != "":
+        elif self.server != "ollama":
-            print("Provider", provider_name, "initialized at", self.server)
+            pretty_print(f"Provider: {provider_name} initialized at {self.server}", color="success")
         self.check_address_format(self.server)
         if not self.is_ip_online(self.server.split(':')[0]):
             raise Exception(f"Server at {self.server} is offline.")
@@ -51,6 +54,7 @@ class Provider:
         Validate if the address is valid IP.
         """
         try:
+            address = address.replace('http://', '')
             ip, port = address.rsplit(":", 1)
             if all(c.lower() in ".:abcdef0123456789" for c in ip):
                 ipaddress.ip_address(ip)
@@ -79,6 +83,8 @@ class Provider:
         """
         Check if an IP address is online by sending a ping request.
         """
+        if ip_address == "127.0.0.1":
+            return True
         param = '-n' if platform.system().lower() == 'windows' else '-c'
         command = ['ping', param, '1', ip_address]
         try:
@@ -86,13 +92,11 @@ class Provider:
             if output.returncode == 0:
                 return True
             else:
-                print("errorcode:", output)
                 return False
         except subprocess.TimeoutExpired:
-            print("timeout")
+            return False
-            return True
         except Exception as e:
-            print(f"is_ip_online error:\n{e}")
+            pretty_print(f"Error with ping request {str(e)}", color="failure")
             return False
 
     def server_fn(self, history, verbose = False):
@@ -114,7 +118,7 @@ class Provider:
                 is_complete = bool(response.json()["is_complete"])
                 time.sleep(2)
         except KeyError as e:
-            raise f"{str(e)}\n\nError occured with server route. Are you using the correct address for the config.ini provider?"
+            raise Exception(f"{str(e)}\n\nError occured with server route. Are you using the correct address for the config.ini provider?") from e
         except Exception as e:
             raise e
         return thought
@@ -138,8 +142,9 @@ class Provider:
             raise Exception("\nOllama connection failed. provider should not be set to ollama if server address is not localhost") from e
         except ollama.ResponseError as e:
             if e.status_code == 404:
-                print(f"Downloading {self.model}...")
+                animate_thinking(f"Downloading {self.model}...")
                 ollama.pull(self.model)
+                self.ollama_fn(history, verbose)
             if "refused" in str(e).lower():
                 raise Exception("Ollama connection failed. is the server running ?") from e
             raise e
@@ -164,8 +169,7 @@ class Provider:
         """
         Use openai to generate text.
         """
-        api_key = self.get_api_key("openai")
+        client = OpenAI(api_key=self.api_key)
-        client = OpenAI(api_key=api_key)
         try:
             response = client.chat.completions.create(
                 model=self.model,
@@ -178,6 +182,24 @@ class Provider:
         except Exception as e:
             raise Exception(f"OpenAI API error: {str(e)}") from e
 
+    def deepseek_fn(self, history, verbose=False):
+        """
+        Use deepseek api to generate text.
+        """
+        client = OpenAI(api_key=self.api_key, base_url="https://api.deepseek.com")
+        try:
+            response = client.chat.completions.create(
+                model="deepseek-chat",
+                messages=history,
+                stream=False
+            )
+            thought = response.choices[0].message.content
+            if verbose:
+                print(thought)
+            return thought
+        except Exception as e:
+            raise Exception(f"Deepseek API error: {str(e)}") from e
+
     def test_fn(self, history, verbose = True):
         """
         This function is used to conduct tests.

sources/router.py

@@ -9,6 +9,7 @@ from sources.agents.agent import Agent
 from sources.agents.code_agent import CoderAgent
 from sources.agents.casual_agent import CasualAgent
 from sources.agents.planner_agent import PlannerAgent
+from sources.agents.browser_agent import BrowserAgent
 from sources.utility import pretty_print
 
 class AgentRouter:
@@ -30,7 +31,7 @@ class AgentRouter:
         else:
             return "cpu"
 
-    def classify_text(self, text: str, threshold: float = 0.5) -> list:
+    def classify_text(self, text: str, threshold: float = 0.4) -> list:
         """
         Classify the text into labels (agent roles).
         Args:
@@ -41,8 +42,8 @@ class AgentRouter:
         """
         first_sentence = None
         for line in text.split("\n"):
-                first_sentence = line.strip()
+            first_sentence = line.strip()
-                break
+            break
         if first_sentence is None:
             first_sentence = text
         result = self.pipeline(first_sentence, self.labels, threshold=threshold)
@@ -61,36 +62,56 @@ class AgentRouter:
         result = self.classify_text(text)
         for agent in self.agents:
             if result["labels"][0] == agent.role:
-                pretty_print(f"Selected agent: {agent.agent_name}", color="warning")
+                pretty_print(f"Selected agent: {agent.agent_name} (roles: {agent.role})", color="warning")
                 return agent
         return None
 
 if __name__ == "__main__":
     agents = [
-        CoderAgent("deepseek-r1:14b", "agent1", "../prompts/coder_agent.txt", "server"),
+        CasualAgent("deepseek-r1:14b", "jarvis", "../prompts/casual_agent.txt", None),
-        CasualAgent("deepseek-r1:14b", "agent2", "../prompts/casual_agent.txt", "server"),
+        BrowserAgent("deepseek-r1:14b", "browser", "../prompts/planner_agent.txt", None),
-        PlannerAgent("deepseek-r1:14b", "agent3", "../prompts/planner_agent.txt", "server")
+        CoderAgent("deepseek-r1:14b", "coder", "../prompts/coder_agent.txt", None)
     ]
     router = AgentRouter(agents)
-    
+    texts = [
-    texts = ["""
+        "Write a python script to check if the device on my network is connected to the internet",
-    Write a python script to check if the device on my network is connected to the internet
+        #"Peut tu écrire un script python qui vérifie si l'appareil sur mon réseau est connecté à internet?",
-    """,
+        #"写一个Python脚本，检查我网络上的设备是否连接到互联网",
-    """
+        "Hey could you search the web for the latest news on the tesla stock market ?",
-    Hey could you search the web for the latest news on the stock market ?
+        #"嘿，你能搜索网页上关于股票市场的最新新闻吗？",
-    """,
+        #"Yo, cherche sur internet comment va tesla en bourse.",
-    """
+        "I would like you to search for weather api and then make an app using this API",
-    hey can you give give a list of the files in the current directory ?
+        #"我想让你搜索天气API，然后用这个API做一个应用程序",
-    """,
+        #"J'aimerais que tu cherche une api météo et que l'utilise pour faire une application",
-    """
+        "Plan a 3-day trip to New York, including flights and hotels.",
-    Make a cool game to illustrate the current relation between USA and europe
+        #"计划一次为期3天的纽约之旅，包括机票和酒店。",
-    """
+        #"Planifie un trip de 3 jours à Paris, y compris les vols et hotels.",
+        "Find me the latest research papers on AI.",
+        #"给我找最新的AI研究论文。",
+        #"Trouve moi les derniers papiers de recherche en IA",
+        "Help me write a C++ program to sort an array",
+        #"帮我写一个C++程序来排序数组",
+        #"Aide moi à faire un programme c++ pour trier une array.",
+        "What’s the weather like today? Oh, and can you find a good weather app?",
+        #"今天天气怎么样？哦，你还能找到一个好的天气应用程序吗？",
+        #"La météo est comment aujourd'hui ? oh et trouve moi une bonne appli météo tant que tu y est.",
+        "Can you debug this Java code? It’s not working.",
+        #"你能调试这段Java代码吗？它不起作用。",
+        #"Peut tu m'aider à debugger ce code java, ça marche pas",
+        "What's the latest brainrot on the internet ?",
+        #"互联网上最新的“脑残”是什么？",
+        #"Quel est la dernière connerie sur internet ?",
+        "i would like to setup a new AI project, index as mark2",
+        #"我想建立一个新的 AI 项目，索引为 Mark2",
+        #"Je voudrais configurer un nouveau projet d'IA, index Mark2",
+        "Hey, can you find the old_project.zip file somewhere on my drive?",
+        #"嘿，你能在我驱动器上找到old_project.zip文件吗？",
+        #"Hé trouve moi le old_project.zip, il est quelque part sur mon disque.",
+        "Tell me a funny story",
+        #"给我讲一个有趣的故事",
+        #"Raconte moi une histoire drole"
     ]
-
     for text in texts:
-        print(text)
+        print("Input text:", text)
-        results = router.classify_text(text)
-        for result in results:
-            print(result["label"], "=>", result["score"])
         agent = router.select_agent(text)
-        print("Selected agent role:", agent.role)
+        print()

sources/speech_to_text.py

@@ -3,9 +3,10 @@ import queue
 import threading
 import numpy as np
 import torch
-from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
 import time
+from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
 import librosa
+import pyaudio
 
 audio_queue = queue.Queue()
 done = False
@@ -14,8 +15,7 @@ class AudioRecorder:
     """
     AudioRecorder is a class that records audio from the microphone and adds it to the audio queue.
     """
-    def __init__(self, format: int, channels: int = 1, rate: int = 4096, chunk: int = 8192, record_seconds: int = 5, verbose: bool = False):
+    def __init__(self, format: int = pyaudio.paInt16, channels: int = 1, rate: int = 4096, chunk: int = 8192, record_seconds: int = 5, verbose: bool = False):
-        import pyaudio
         self.format = format
         self.channels = channels
         self.rate = rate

sources/text_to_speech.py

@@ -4,6 +4,7 @@ import soundfile as sf
 import subprocess
 import re
 import platform
+from sys import modules
 
 class Speech():
     """
@@ -24,7 +25,7 @@ class Speech():
         self.voice = self.voice_map[language][2]
         self.speed = 1.2
 
-    def speak(self, sentence: str, voice_number: int = 1):
+    def speak(self, sentence: str, voice_number: int = 1 , audio_file: str = 'sample.wav'):
         """
         Convert text to speech using an AI model and play the audio.
 
@@ -38,17 +39,17 @@ class Speech():
             sentence, voice=self.voice,
             speed=self.speed, split_pattern=r'\n+'
         )
-        for i, (gs, ps, audio) in enumerate(generator):
+        for i, (_, _, audio) in enumerate(generator):
-            audio_file = 'sample.wav'
+            if 'ipykernel' in modules: #only display in jupyter notebook.
-            display(Audio(data=audio, rate=24000, autoplay=i==0), display_id=False)
+                display(Audio(data=audio, rate=24000, autoplay=i==0), display_id=False)
             sf.write(audio_file, audio, 24000) # save each audio file
             if platform.system().lower() == "windows":
                 import winsound
                 winsound.PlaySound(audio_file, winsound.SND_FILENAME)
-            elif platform.system().lower() == "linux":
+            elif platform.system().lower() == "darwin":  # macOS
-                subprocess.call(["aplay", audio_file])
-            else:
                 subprocess.call(["afplay", audio_file])
+            else: # linux or other.
+                subprocess.call(["aplay", audio_file])
 
     def replace_url(self, url: re.Match) -> str:
         """

sources/tools/searxSearch.py

@@ -50,7 +50,6 @@ class searxSearch(Tools):
         """Check all links, one by one."""
         # TODO Make it asyncromous or smth
         statuses = []
-        print("Web scrawl to verify links accessibilty...")
         for i, link in enumerate(links):
             status = self.link_valid(link)
             statuses.append(status)
@@ -90,9 +89,11 @@ class searxSearch(Tools):
                     title = article.find('h3').text.strip() if article.find('h3') else "No Title"
                     description = article.find('p', class_='content').text.strip() if article.find('p', class_='content') else "No Description"
                     results.append(f"Title:{title}\nSnippet:{description}\nLink:{url}")
+            if len(results) == 0:
+                return "No search results, web search failed."
             return "\n\n".join(results)  # Return results as a single string, separated by newlines
         except requests.exceptions.RequestException as e:
-            return f"Error during search: {str(e)}"
+            raise Exception("\nSearxng search failed. did you run start_services.sh? is docker still running?") from e
 
     def execution_failure_check(self, output: str) -> bool:
         """

sources/tools/tools.py

@@ -118,10 +118,8 @@ class Tools():
         save_path_file = os.path.basename(save_path)
         directory = os.path.join(self.current_dir, save_path_dir)
         if directory and not os.path.exists(directory):
-            print(f"Creating directory: {directory}")
             os.makedirs(directory)
         for block in blocks:
-            print(f"Saving code block to: {save_path}")
             with open(os.path.join(directory, save_path_file), 'w') as f:
                 f.write(block)
     

sources/tools/webSearch.py

@@ -58,7 +58,6 @@ class webSearch(Tools):
         """Check all links, one by one."""
         # TODO Make it asyncromous or smth
         statuses = []
-        print("Workers started, scrawling the web...")
         for i, link in enumerate(links):
             status = self.link_valid(link)
             statuses.append(status)

start_services.cmd

@@ -0,0 +1,7 @@
+@echo off
+
+REM Up the provider in windows
+start ollama serve
+
+REM Up Docker
+cd searxng && docker compose up