Feat: prompt change for casual agent

Update text_to_speech.py ,  browser.py .

.pre-commit-config.yaml

@@ -0,0 +1,9 @@
+repos:
+  - repo: local
+    hooks:
+      - id: trufflehog
+        name: TruffleHog
+        description: Detect secrets in your data.
+        entry: bash -c 'trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail --no-update'
+        language: system
+        stages: ["commit", "push"]

CONTRIBUTING.md

@@ -74,11 +74,13 @@ Push your changes to your fork and submit a pull request to the main branch of t
 
 Here are some high-priority tasks and areas where we need contributions:
 
-- Web Browsing: Implement autonomous web browsing capabilities for the assistant.
+- Web Browsing: Improve the autonomous web browsing capabilities for the assistant.
-- Multi-Agent System: Enhance the multi-agent functionality on the dev branch.
+- Graphical interface, a web graphical interface. (please ask first)
-- Memory & Recovery: Improve conversation compression.
+- Multi-Agent System: Enhance the planner agent for divide and conqueer for task (please ask first).
 - New Tools: Add support for additional programming languages or APIs.
-- Testing: Write comprehensive tests for existing and new features.
+- Multi-language support for Text to speech & speech to text (english, chinese, spanish first)
+- Testing: Write comprehensive tests for existing features.
+- Better readme image: make a better readme image (robot whale that use tools. Ghibli or anime style, inspiration could be https://sakana.ai/assets/ai-scientist/cover.jpeg)
 
 
 If you're unsure where to start, feel free to reach out by opening an issue or joining our community discussions.

Dockerfile

@@ -0,0 +1,26 @@
+# Use official Python 3.11 image as the base
+FROM python:3.11
+
+# Set working directory
+WORKDIR /app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    gcc \
+    g++ \
+    gfortran \
+    libportaudio2 \
+    portaudio19-dev \
+    ffmpeg \
+    libavcodec-dev \
+    libavformat-dev \
+    libavutil-dev \
+    chromium \
+    chromium-driver \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN pip cache purge
+
+COPY . .
+
+RUN BLIS_ARCH=generic pip install --no-cache-dir -r requirements.txt

README.md

@@ -4,12 +4,19 @@
 
 **A fully local alternative to Manus AI**, a voice-enabled AI assistant that codes, explores your filesystem, browse the web and correct it's mistakes all without sending a byte of data to the cloud. Built with reasoning models like DeepSeek R1, this autonomous agent runs entirely on your hardware, keeping your data private.
 
-[![Visit AgenticSeek](https://img.shields.io/static/v1?label=Website&message=AgenticSeek&color=blue&style=flat-square)](https://fosowl.github.io/agenticSeek.html) ![License](https://img.shields.io/badge/license-GPL--3.0-green) ![Discord](https://img.shields.io/badge/Discord-Join%20Us-7289DA?logo=discord&logoColor=white)
+[![Visit AgenticSeek](https://img.shields.io/static/v1?label=Website&message=AgenticSeek&color=blue&style=flat-square)](https://fosowl.github.io/agenticSeek.html) ![License](https://img.shields.io/badge/license-GPL--3.0-green) [![Discord](https://img.shields.io/badge/Discord-Join%20Us-7289DA?logo=discord&logoColor=white)](https://discord.gg/4Ub2D6Fj)
+
 > 🛠️ **Work in Progress** – Looking for contributors!
 
 ![alt text](./media/whale_readme.jpg)
 
+> *Do a web search to find tech startup in Japan working on cutting edge AI research*
 
+> *Make a snake game in Python*
+
+> *Scan my network with nmap, find out who is connected?*
+
+> *Hey can you find where is contract.pdf*?
 
 ## Features:
 
@@ -31,7 +38,7 @@
 
 - **Memory**: Remembers what’s useful, your preferences and past sessions conversation.
 
-- **Web Browsing**: Autonomous web navigation is underway.
+- **Web Browsing**: Autonomous web navigation.
 
 
 ### Searching the web with agenticSeek :
@@ -44,11 +51,16 @@
 
 ## **Installation**
 
-### 1️⃣ **Clone the repository**
+Make sure you have chrome driver, docker and python3.10 (or newer) installed.
+
+For issues related to chrome driver, see the **Chromedriver** section.
+
+### 1️⃣ **Clone the repository and setup**
 
 ```sh
 git clone https://github.com/Fosowl/agenticSeek.git
 cd agenticSeek
+mv .env.example .env
 ```
 
 ### 2️ **Create a virtual env**
@@ -84,10 +96,10 @@ python3 setup.py install
 
 Make sure you have [Ollama](https://ollama.com/) installed.
 
-Download the `deepseek-r1:7b` model from [DeepSeek](https://deepseek.com/models)
+Download the `deepseek-r1:14b` model from [DeepSeek](https://deepseek.com/models)
 
 ```sh
-ollama pull deepseek-r1:7b
+ollama pull deepseek-r1:14b
 ```
 
 ### 2️ **Run the Assistant (Ollama)**  
@@ -97,21 +109,22 @@ Start the ollama server
 ollama serve
 ```
 
-Change the config.ini file to set the provider_name to `ollama` and provider_model to `deepseek-r1:7b`
+Change the config.ini file to set the provider_name to `ollama` and provider_model to `deepseek-r1:14b`
 
-NOTE: `deepseek-r1:7b`is an example, use a bigger model if your hardware allow it.
+NOTE: `deepseek-r1:14b`is an example, use a bigger model if your hardware allow it.
 
 ```sh
 [MAIN]
 is_local = True
 provider_name = ollama
-provider_model = deepseek-r1:7b
+provider_model = deepseek-r1:14b
+provider_server_address = 127.0.0.1:11434
 ```
 
 start all services :
 
 ```sh
-./start_services.sh
+sudo ./start_services.sh
 ```
 
 Run the assistant:
@@ -120,6 +133,77 @@ Run the assistant:
 python3 main.py
 ```
 
+*See the **Usage** section if you don't understand how to use it*
+
+*See the **Known issues** section if you are having issues*
+
+*See the **Run with an API** section if your hardware can't run deepseek locally*
+
+---
+
+## Usage
+
+Warning: currently the system that choose the best AI agent routing system will work poorly with non-english text. This is because the agent routing currently use a model that was trained on english text. We are working hard to fix this. Please use english for now.
+
+
+Make sure the services are up and running with `./start_services.sh` and run the agenticSeek with `python3 main.py`
+
+```sh
+sudo ./start_services.sh
+python3 main.py
+```
+
+You will be prompted with `>>> `
+This indicate agenticSeek await you type for instructions.
+You can also use speech to text by setting `listen = True` in the config.
+
+Here are some example usage:
+
+### Coding/Bash
+
+> *Help me with matrix multiplication in Golang*
+
+> *Scan my network with nmap, find if any suspicious devices is connected*
+
+> *Make a snake game in python*
+
+### Web search
+
+> *Do a web search to find cool tech startup in Japan working on cutting edge AI research*
+
+> *Can you find on the internet who created agenticSeek?*
+
+> *Can you find on which website I can buy a rtx 4090 for cheap*
+
+### File system
+
+> *Hey can you find where is million_dollars_contract.pdf i lost it*
+
+> *Show me how much space I have left on my disk*
+
+> *Find and read the README.md and follow the install instruction*
+
+### Casual
+
+> *Tell me a joke*
+
+> *Where is flight ABC777 ? my mom is on that plane*
+
+> *what is the meaning of life ?*
+
+
+After you type your query, agenticSeek will allocate the best agent for the task.
+
+Because this is an early prototype, the agent routing system might not always allocate the right agent based on your query.
+
+Therefore, you should be very explicit in what you want and how the AI might proceed for example if you want it to conduct a web search, do not say:
+
+`Do you know some good countries for solo-travel?`
+
+Instead, ask:
+
+`Do a web search and find out which are the best country for solo-travel`
+
 ---
 
 ## **Run the LLM on your own server**  
@@ -128,6 +212,8 @@ If you have a powerful computer or a server that you can use, but you want to us
 
 ### 1️⃣  **Set up and start the server scripts** 
 
+You need to have ollama installed on the server (We will integrate VLLM and llama.cpp soon).
+
 On your "server" that will run the AI model, get the ip address
 
 ```sh
@@ -139,7 +225,7 @@ Note: For Windows or macOS, use ipconfig or ifconfig respectively to find the IP
 Clone the repository and then, run the script `stream_llm.py` in `server/`
 
 ```sh
-python3 server_ollama.py
+python3 server_ollama.py --model "deepseek-r1:32b"
 ```
 
 ### 2️⃣ **Run it** 
@@ -148,7 +234,7 @@ Now on your personal computer:
 
 Clone the repository.
 
-Change the `config.ini` file to set the `provider_name` to `server` and `provider_model` to `deepseek-r1:7b`.
+Change the `config.ini` file to set the `provider_name` to `server` and `provider_model` to `deepseek-r1:14b`.
 Set the `provider_server_address` to the ip address of the machine that will run the model.
 
 ```sh
@@ -162,7 +248,7 @@ provider_server_address = x.x.x.x:5000
 Run the assistant:
 
 ```sh
-./start_services.sh
+sudo ./start_services.sh
 python3 main.py
 ```
 
@@ -183,12 +269,39 @@ provider_server_address = 127.0.0.1:5000 # can be set to anything, not used
 Run the assistant:
 
 ```sh
-./start_services.sh
+sudo ./start_services.sh
 python3 main.py
 ```
 
 ---
 
+
+
+## Speech to Text
+
+The speech-to-text functionality is disabled by default. To enable it, set the listen option to True in the config.ini file:
+
+```
+listen = True
+```
+
+When enabled, the speech-to-text feature listens for a trigger keyword, which is the agent's name, before it begins processing your input. You can customize the agent's name by updating the `agent_name` value in the *config.ini* file:
+
+```
+agent_name = Friday
+```
+
+For optimal recognition, we recommend using a common English name like "John" or "Emma" as the agent name
+
+Once you see the transcript start to appear, say the agent's name aloud to wake it up (e.g., "Friday").
+
+Speak your query clearly.
+
+End your request with a confirmation phrase to signal the system to proceed. Examples of confirmation phrases include:
+```
+"do it", "go ahead", "execute", "run", "start", "thanks", "would ya", "please", "okay?", "proceed", "continue", "go on", "do that", "go it", "do you understand?"
+```
+
 ## Providers
 
 The table below show the available providers:
@@ -212,13 +325,37 @@ provider_server_address = 127.0.0.1:5000
 ```
 `is_local`: should be True for any locally running LLM, otherwise False.
 
-`provider_name`: Select the provider to use by its name, see the provider list above.
+`provider_name`: Select the provider to use by it's name, see the provider list above.
 
 `provider_model`: Set the model to use by the agent.
 
 `provider_server_address`: can be set to anything if you are not using the server provider.
 
+# Known issues
+
+## Chromedriver Issues
+
+**Known error #1:** *chromedriver mismatch*
+
+`Exception: Failed to initialize browser: Message: session not created: This version of ChromeDriver only supports Chrome version 113
+Current browser version is 134.0.6998.89 with binary path`
+
+This happen if there is a mismatch between your browser and chromedriver version.
+
+You need to navigate to download the latest version:
+
+https://developer.chrome.com/docs/chromedriver/downloads
+
+If you're using Chrome version 115 or newer go to:
+
+https://googlechromelabs.github.io/chrome-for-testing/
+
+And download the chromedriver version matching your OS.
+
+![alt text](./media/chromedriver_readme.png)
+
 ## FAQ
+
 **Q: What hardware do I need?**  
 
 7B Model: GPU with 8GB VRAM.
@@ -233,10 +370,6 @@ Deepseek R1 excels at reasoning and tool use for its size. We think it’s a sol
 
 Ensure Ollama is running (`ollama serve`), your `config.ini` matches your provider, and dependencies are installed. If none work feel free to raise an issue.
 
-**Q: How to join the discord ?**
-
-Ask in the Community section for an invite.
-
 **Q: Can it really run 100% locally?**  
 
 Yes with Ollama or Server providers, all speech to text, LLM and text to speech model run locally. Non-local options (OpenAI or others API) are optional.
@@ -253,6 +386,8 @@ It's not but we prioritizes local execution and privacy over cloud based approac
 
 We’re looking for developers to improve AgenticSeek! Check out open issues or discussion.
 
+[![Star History Chart](https://api.star-history.com/svg?repos=Fosowl/agenticSeek&type=Date)](https://www.star-history.com/#Fosowl/agenticSeek&Date)
+
 ## Authors:
  > [Fosowl](https://github.com/Fosowl)
  > [steveh8758](https://github.com/steveh8758) 

config.ini

@@ -2,10 +2,10 @@
 is_local = True
 provider_name = ollama
 provider_model = deepseek-r1:14b
-provider_server_address = 127.0.0.1:5000
+provider_server_address = 127.0.0.1:11434
 agent_name = Friday
 recover_last_session = True
 save_session = False
 speak = True
 listen = False
-work_dir =  /Users/mlg/Documents/A-project/AI/Agents/agenticSeek/ai_workplace
+work_dir =  /Users/mlg/Documents/ai_workplace

main.py

@@ -29,26 +29,18 @@ def main():
                                    server_address=config["MAIN"]["provider_server_address"])
 
     agents = [
-        CasualAgent(model=config["MAIN"]["provider_model"],
+        CasualAgent(name=config["MAIN"]["agent_name"],
-                       name=config["MAIN"]["agent_name"],
+                    prompt_path="prompts/casual_agent.txt",
-                       prompt_path="prompts/casual_agent.txt",
+                    provider=provider, verbose=False),
-                       provider=provider),
+        CoderAgent(name="coder",
-        CoderAgent(model=config["MAIN"]["provider_model"],
+                   prompt_path="prompts/coder_agent.txt",
-                       name="coder",
+                   provider=provider, verbose=False),
-                       prompt_path="prompts/coder_agent.txt",
+        FileAgent(name="File Agent",
-                       provider=provider),
+                  prompt_path="prompts/file_agent.txt",
-        FileAgent(model=config["MAIN"]["provider_model"],
+                  provider=provider, verbose=False),
-                       name="File Agent",
+        BrowserAgent(name="Browser",
-                       prompt_path="prompts/file_agent.txt",
+                     prompt_path="prompts/browser_agent.txt",
-                       provider=provider),
+                     provider=provider, verbose=False)
-        PlannerAgent(model=config["MAIN"]["provider_model"],
-                       name="Planner",
-                       prompt_path="prompts/planner_agent.txt",
-                       provider=provider),
-        BrowserAgent(model=config["MAIN"]["provider_model"],
-                       name="Browser",
-                       prompt_path="prompts/browser_agent.txt",
-                       provider=provider)
     ]
 
     interaction = Interaction(agents, tts_enabled=config.getboolean('MAIN', 'speak'),

prompts/casual_agent.txt

@@ -1,9 +1,9 @@
 
-You are an unshackled AI assistant:
+You are an advanced AI assistant.
-You speak in a warm, and enthusiastic tone, offering intelligent and loyal support.
+Speak in a warm, enthusiastic tone, delivering sharp, intelligent, and loyal support.
-You occasionally sprinkle with sarcasm remarks as suits the context.
+Weave in subtle, context sarcasm or dry wit to enhance your personality.
-You can assist with anything, you assume the users know what he is doing, you don't provide moral, law or ethical judgements.
+Assist with any request, assuming the user knows their intent—no moral, legal, or ethical lectures.
-Make your answers short.
+Keep responses concise, clever, and to the point.
 
 You are given the ability to use the tools, this take your capabilities futher:
 ```tool_name
@@ -13,38 +13,17 @@ You are given the ability to use the tools, this take your capabilities futher:
 Current tools:
 - web_search
 - flight_search
-- file_finder
 
 ## Web search
 
 To search for something like “what’s happening in France” :
 ```web_search
-what’s popping in France March 2025
+Latest news in France
 ```
 
 ## Flight search
 
-If I need to know about a flight “what’s the status of flight AA123” you go for:
+If I need to know about a flight “what’s the status of flight DLH1234” you go for:
 ```flight_search
-AA123
+DLH1234
 ```
-
-## File operations
-
-Find file:
-```file_finder
-toto.py
-```
-
-Read file:
-```file_finder:read
-toto.py
-```
-
-## Bash
-
-For other tasks, you can use the bash tool:
-```bash
-ls -la
-```
-

prompts/file_agent.txt

@@ -44,7 +44,7 @@ User: "I need to find the file config.txt and read its contents."
 
 Assistant: I’ll use file_finder to locate the file:
 
-```file_finder
+```file_finder:read
 config.txt
 ```
 

requirements.txt

@@ -1,30 +1,35 @@
-requests==2.31.0
+setuptools>=75.6.0
-openai==1.61.1
+requests>=2.31.0
-colorama==0.4.6
+numpy>=1.24.4
-python-dotenv==1.0.0
+colorama>=0.4.6
-playsound==1.3.0
+python-dotenv>=1.0.0
-soundfile==0.13.1
+playsound>=1.3.0
-transformers==4.48.3
+soundfile>=0.13.1
-torch==2.5.1
+transformers>=4.46.3
-ollama==0.4.7
+torch>=2.4.1
-scipy==1.15.1
+python-dotenv>=1.0.0
-kokoro==0.7.12
+ollama>=0.4.7
-flask==3.1.0
+scipy>=1.9.3
-soundfile==0.13.1
+kokoro>=0.7.12
-protobuf==3.20.3
+soundfile>=0.13.1
-termcolor==2.5.0
+protobuf>=3.20.3
-ipython==8.34.0
+termcolor>=2.4.0
-gliclass==0.1.8
+ipython>=8.13.0
-pyaudio==0.2.14
+pyaudio>=0.2.14
-librosa==0.10.2.post1
+librosa>=0.10.2.post1
-selenium==4.29.0
+selenium>=4.27.1
-markdownify==1.1.0
+markdownify>=1.1.0
+text2emotion>=0.0.5
+langid>=1.1.6
+chromedriver-autoinstaller>=0.6.4
 httpx>=0.27,<0.29
 anyio>=3.5.0,<5
 distro>=1.7.0,<2
 jiter>=0.4.0,<1
 sniffio
 tqdm>4
+# for api provider
+openai
 # if use chinese
 ordered_set
 pypinyin

scripts/linux_install.sh

@@ -5,6 +5,19 @@ echo "Starting installation for Linux..."
 # Update package list
 sudo apt-get update
 
+pip install --upgrade pip
+
+# install pyaudio
+pip install pyaudio
+# make sure essential tool are installed
+sudo apt install python3-dev python3-pip python3-wheel build-essential
+# install port audio
+sudo apt-get install portaudio19-dev python-pyaudio python3-pyaudio
+# install wheel
+pip install --upgrade pip setuptools wheel
+# install docker compose
+sudo apt install docker-compose
+
 # Install Python dependencies from requirements.txt
 pip3 install -r requirements.txt
 

searxng/.searxng.env

@@ -1 +0,0 @@
-SEARXNG_BASE_URL="http://127.0.0.1:8080"

searxng/docker-compose.yml

@@ -1,3 +1,4 @@
+version: '3'
 services:
   redis:
     container_name: redis

searxng/setup_searxng.sh

@@ -85,17 +85,6 @@ else
     exit 1
 fi
 
-# Stop containers
-echo "Stopping containers to apply security settings..."
-docker-compose down
-
-# Start containers again with secure settings
-echo "Deploying SearXNG with secure settings..."
-if ! docker-compose up -d; then
-    echo "Error: Failed to deploy SearXNG. Check logs with 'docker compose logs'."
-    exit 1
-fi
-
 # Display status and access instructions
 echo "SearXNG setup complete!"
 docker ps -a --filter "name=searxng" --filter "name=redis"

server/Dockerfile

@@ -0,0 +1,14 @@
+FROM ubuntu:20.04
+
+WORKDIR /app
+
+RUN apt-get update && \
+    apt-get install -y python3 python3-pip && \
+    apt-get clean && \
+    rm -rf /var/lib/apt/lists/*
+
+COPY requirements.txt .
+
+RUN pip3 install --no-cache-dir -r requirements.txt
+
+CMD ["python3", "--version"]

server/config.json

@@ -1,30 +0,0 @@
-{
-    "model_name": "deepseek-r1:14b",
-    "known_models": [
-        "qwq:32b",
-        "deepseek-r1:1.5b",
-        "deepseek-r1:7b",
-        "deepseek-r1:14b",
-        "deepseek-r1:32b",
-        "deepseek-r1:70b",
-        "deepseek-r1:671b",
-        "deepseek-coder:1.3b",
-        "deepseek-coder:6.7b",
-        "deepseek-coder:33b",
-        "llama2-uncensored:7b",
-        "llama2-uncensored:70b",
-        "llama3.1:8b",
-        "llama3.1:70b",
-        "llama3.3:70b",
-        "llama3:8b",
-        "llama3:70b",
-        "i4:14b",
-        "mistral:7b",
-        "mistral:70b",
-        "mistral:33b",
-        "qwen1:7b",
-        "qwen1:14b",
-        "qwen1:32b",
-        "qwen1:70b"
-    ]
-}

server/requirements.txt

@@ -0,0 +1,2 @@
+flask>=2.3.0
+ollama>=0.4.7

server/server_ollama.py

@@ -1,44 +1,37 @@
+#!/usr/bin python3
+
+# NOTE this script is temporary and will be improved
+
 from flask import Flask, jsonify, request
 import threading
 import ollama
 import logging
-import json
+import argparse
 
 log = logging.getLogger('werkzeug')
 log.setLevel(logging.ERROR)
 
+parser = argparse.ArgumentParser(description='AgenticSeek server script')
+parser.add_argument('--model', type=str, help='Model to use. eg: deepseek-r1:14b', required=True)
+args = parser.parse_args()
+
 app = Flask(__name__)
 
+model = args.model
+
 # Shared state with thread-safe locks
-class Config:
-    def __init__(self):
-        self.model = None
-        self.known_models = []
-        self.allowed_models = []
-        self.model_name = None
-
-    def load(self):
-        with open('config.json', 'r') as f:
-            data = json.load(f)
-            self.known_models = data['known_models']
-            self.model_name = data['model_name']
-
-    def validate_model(self, model):
-        if model not in self.known_models:
-            raise ValueError(f"Model {model} is not known")
-
 class GenerationState:
     def __init__(self):
         self.lock = threading.Lock()
         self.last_complete_sentence = ""
         self.current_buffer = ""
         self.is_generating = False
-        self.model = None
 
 state = GenerationState()
 
-def generate_response(history):  # Only takes history as an argument
+def generate_response(history, model):
     global state
+    print("using model:::::::", model)
     try:
         with state.lock:
             state.is_generating = True
@@ -46,18 +39,21 @@ def generate_response(history):  # Only takes history as an argument
             state.current_buffer = ""
 
         stream = ollama.chat(
-            model=state.model,  # Access state.model directly
+            model=model,
             messages=history,
             stream=True,
         )
+
         for chunk in stream:
             content = chunk['message']['content']
             print(content, end='', flush=True)
+
             with state.lock:
                 state.current_buffer += content
+
     except ollama.ResponseError as e:
         if e.status_code == 404:
-            ollama.pull(state.model)
+            ollama.pull(model)
         with state.lock:
             state.is_generating = False
         print(f"Error: {e}")
@@ -75,8 +71,8 @@ def start_generation():
             return jsonify({"error": "Generation already in progress"}), 400
 
         history = data.get('messages', [])
-        # Pass only history to the thread
+        # Start generation in background thread
-        threading.Thread(target=generate_response, args=(history,)).start()  # Note the comma to make it a single-element tuple
+        threading.Thread(target=generate_response, args=(history, model)).start()
     return jsonify({"message": "Generation started"}), 202
 
 @app.route('/get_updated_sentence')
@@ -89,8 +85,4 @@ def get_updated_sentence():
         })
 
 if __name__ == '__main__':
-    config = Config()
+    app.run(host='0.0.0.0', threaded=True, debug=True, port=5000)
-    config.load()
-    config.validate_model(config.model_name)
-    state.model = config.model_name
-    app.run(host='0.0.0.0', port=5000, debug=False, threaded=True)

setup.py

@@ -8,32 +8,34 @@ setup(
     version="0.1.0",
     author="Fosowl",
     author_email="mlg.fcu@gmail.com",
-    description="A Python project for agentic search and processing",
+    description="The open, local alternative to ManusAI",
     long_description=long_description,
     long_description_content_type="text/markdown",
     url="https://github.com/Fosowl/agenticSeek",
     packages=find_packages(),
     include_package_data=True,
     install_requires=[
-        "requests==2.31.0",
+        "requests>=2.31.0",
-        "openai==1.61.1",
+        "openai",
-        "colorama==0.4.6",
+        "colorama>=0.4.6",
-        "python-dotenv==1.0.0",
+        "python-dotenv>=1.0.0",
-        "playsound==1.3.0",
+        "playsound>=1.3.0",
-        "soundfile==0.13.1",
+        "soundfile>=0.13.1",
-        "transformers==4.48.3",
+        "transformers>=4.46.3",
-        "torch==2.5.1",
+        "torch>=2.4.1",
-        "ollama==0.4.7",
+        "ollama>=0.4.7",
-        "scipy==1.15.1",
+        "scipy>=1.9.3",
-        "kokoro==0.7.12",
+        "kokoro>=0.7.12",
-        "flask==3.1.0",
+        "flask>=3.1.0",
-        "protobuf==3.20.3",
+        "protobuf>=3.20.3",
-        "termcolor==2.5.0",
+        "termcolor>=2.5.0",
-        "gliclass==0.1.8",
+        "ipython>=8.34.0",
-        "ipython==8.34.0",
+        "librosa>=0.10.2.post1",
-        "librosa==0.10.2.post1",
+        "selenium>=4.29.0",
-        "selenium==4.29.0",
+        "markdownify>=1.1.0",
-        "markdownify==1.1.0",
+        "text2emotion>=0.0.5",
+        "python-dotenv>=1.0.0",
+        "langid>=1.1.6",
         "httpx>=0.27,<0.29",
         "anyio>=3.5.0,<5",
         "distro>=1.7.0,<2",
@@ -59,5 +61,5 @@ setup(
         "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
         "Operating System :: OS Independent",
     ],
-    python_requires=">=3.6",
+    python_requires=">=3.9",
 )

sources/agents/agent.py

@@ -30,15 +30,24 @@ class Agent():
     """
     An abstract class for all agents.
     """
-    def __init__(self, model: str,
+    def __init__(self, name: str,
-                       name: str,
                        prompt_path:str,
                        provider,
-                       recover_last_session=True) -> None:
+                       recover_last_session=True,
+                       verbose=False) -> None:
+        """
+        Args:
+            name (str): Name of the agent.
+            prompt_path (str): Path to the prompt file for the agent.
+            provider: The provider for the LLM.
+            recover_last_session (bool, optional): Whether to recover the last conversation. 
+            verbose (bool, optional): Enable verbose logging if True. Defaults to False.
+        """
+            
         self.agent_name = name
         self.role = None
+        self.type = None
         self.current_directory = os.getcwd()
-        self.model = model
         self.llm = provider 
         self.memory = Memory(self.load_prompt(prompt_path),
                                 recover_last_session=recover_last_session,
@@ -46,6 +55,7 @@ class Agent():
         self.tools = {}
         self.blocks_result = []
         self.last_answer = ""
+        self.verbose = verbose
     
     @property
     def get_tools(self) -> dict:
@@ -93,12 +103,12 @@ class Agent():
         end_idx = text.rfind(end_tag)+8
         return text[start_idx:end_idx]
     
-    def llm_request(self, verbose = False) -> Tuple[str, str]:
+    def llm_request(self) -> Tuple[str, str]:
         """
         Ask the LLM to process the prompt and return the answer and the reasoning.
         """
         memory = self.memory.get()
-        thought = self.llm.respond(memory, verbose)
+        thought = self.llm.respond(memory, self.verbose)
 
         reasoning = self.extract_reasoning_text(thought)
         answer = self.remove_reasoning_text(thought)

sources/agents/browser_agent.py

@@ -5,32 +5,48 @@ from sources.utility import pretty_print, animate_thinking
 from sources.agents.agent import Agent
 from sources.tools.searxSearch import searxSearch
 from sources.browser import Browser
+from datetime import date
+from typing import List, Tuple
 
 class BrowserAgent(Agent):
-    def __init__(self, model, name, prompt_path, provider):
+    def __init__(self, name, prompt_path, provider, verbose=False):
         """
         The Browser agent is an agent that navigate the web autonomously in search of answer
         """
-        super().__init__(model, name, prompt_path, provider)
+        super().__init__(name, prompt_path, provider, verbose)
         self.tools = {
             "web_search": searxSearch(),
         }
-        self.role = "deep research and web search"
+        self.role = "Web search and navigation"
+        self.type = "browser_agent"
         self.browser = Browser()
-        self.browser.go_to("https://github.com/")
+        self.current_page = ""
         self.search_history = []
         self.navigable_links = []
         self.notes = []
+        self.date = self.get_today_date()
     
+    def get_today_date(self) -> str:
+        """Get the date"""
+        date_time = date.today()
+        return date_time.strftime("%B %d, %Y")
 
-    def extract_links(self, search_result: str):
+    def extract_links(self, search_result: str) -> List[str]:
+        """Extract all links from a sentence."""
         pattern = r'(https?://\S+|www\.\S+)'
         matches = re.findall(pattern, search_result)
-        trailing_punct = ".,!?;:"
+        trailing_punct = ".,!?;:)"
         cleaned_links = [link.rstrip(trailing_punct) for link in matches]
         return self.clean_links(cleaned_links)
     
-    def clean_links(self, links: list):
+    def extract_form(self, text: str) -> List[str]:
+        """Extract form written by the LLM in format [input_name](value)"""
+        inputs = []
+        matches = re.findall(r"\[\w+\]\([^)]+\)", text)
+        return matches
+        
+    def clean_links(self, links: List[str]) -> List[str]:
+        """Ensure no '.' at the end of link"""
         links_clean = []
         for link in links:
             link = link.strip()
@@ -40,62 +56,99 @@ class BrowserAgent(Agent):
                 links_clean.append(link)
         return links_clean
 
-    def get_unvisited_links(self):
+    def get_unvisited_links(self) -> List[str]:
         return "\n".join([f"[{i}] {link}" for i, link in enumerate(self.navigable_links) if link not in self.search_history])
 
-    def make_newsearch_prompt(self, user_prompt: str, search_result: dict):
+    def make_newsearch_prompt(self, user_prompt: str, search_result: dict) -> str:
         search_choice = self.stringify_search_results(search_result)
         return f"""
         Based on the search result:
         {search_choice}
         Your goal is to find accurate and complete information to satisfy the user’s request.
         User request: {user_prompt}
-        To proceed, choose a relevant link from the search results. Announce your choice by saying: "I want to navigate to <link>."
+        To proceed, choose a relevant link from the search results. Announce your choice by saying: "I want to navigate to <link>"
         Do not explain your choice.
         """
     
-    def make_navigation_prompt(self, user_prompt: str, page_text: str):
+    def make_navigation_prompt(self, user_prompt: str, page_text: str) -> str:
         remaining_links = self.get_unvisited_links() 
-        remaining_links_text = remaining_links if remaining_links is not None else "No links remaining, proceed with a new search." 
+        remaining_links_text = remaining_links if remaining_links is not None else "No links remaining, do a new search." 
-        return f"""
+        inputs_form = self.browser.get_form_inputs()
-        \nYou are currently browsing the web. Not the user, you are the browser.
+        inputs_form_text = '\n'.join(inputs_form)
 
-        Page content:
+        return f"""
+        You are a web browser.
+        You are currently on this webpage:
         {page_text}
 
-        You can navigate to these links:
+        You can navigate to these navigation links:
-        {remaining_links}
+        {remaining_links_text}
 
-        If no link seem appropriate, please say "GO_BACK".
+        Your task:
-        Remember, you seek the information the user want.
+        1. Decide if the current page answers the user’s query: {user_prompt}
-        The user query was : {user_prompt}
+          - If it does, take notes of the useful information, write down source, link or reference, then move to a new page.
-        You must choose a link (write it down) to navigate to, or go back.
+          - If it does and you are 100% certain that it provide a definive answer, say REQUEST_EXIT
-        For exemple you can say: i want to go to www.wikipedia.org/cats
+          - If it doesn’t, say: Error: This page does not answer the user’s query then go back or navigate to another link.
-        Always end with a sentence that summarize when useful information is found for exemple:
+        2. Navigate by either: 
-        Summary: According to https://karpathy.github.io/ LeCun net is the earliest real-world application of a neural net"
+          - Navigate to a navigation links (write the full URL, e.g., www.example.com/cats).
-        Do not say "according to this page", always write down the whole link.
+          - If no link seems helpful, say: GO_BACK.
-        If a website does not have usefull information say Error, for exemple:
+        3. Fill forms on the page:
-        Error: This forum does not discus anything that can answer the user query
+          - If user give you informations that help you fill form, fill it.
-        Do not explain your choice, be short, concise.
+          - If you don't know how to fill a form, leave it empty.
+          - You can fill a form using [form_name](value).
+        
+        Recap of note taking:
+        If useful -> Note: [Briefly summarize the key information or task you conducted.]
+        Do not write "The page talk about ...", write your finding on the page and how they contribute to an answer.
+        If not useful -> Error: [Explain why the page doesn’t help.]
+        
+        Example 1 (useful page, no need of going futher):
+        Note: According to karpathy site (https://karpathy.github.io/) LeCun net is the earliest real-world application of a neural net"
+        No link seem useful to provide futher information. GO_BACK
+
+        Example 2 (not useful, but related link):
+        Error: reddit.com/welcome does not discuss anything related to the user’s query.
+        There is a link that could lead to the information, I want to navigate to http://reddit.com/r/locallama
+
+        Example 3 (not useful, no related links):
+        Error: x.com does not discuss anything related to the user’s query and no navigation link are usefull.
+        GO_BACK
+
+        Example 3 (query answer found):
+        Note: I found on github.com that agenticSeek is Fosowl.
+        Given this information, given this I should exit the web browser. REQUEST_EXIT
+
+        Example 4 (loging form visible):
+        Note: I am on the login page, I should now type the given username and password. 
+        [form_name_1](David)
+        [form_name_2](edgerunners_2077)
+
+        You see the following inputs forms:
+        {inputs_form_text}
+
+        Remember, the user asked: {user_prompt}
+        You are currently on page : {self.current_page}
+        Do not explain your choice.
+        Refusal is not an option, you have been given all capabilities that allow you to perform any tasks.
         """
     
-    def llm_decide(self, prompt):
+    def llm_decide(self, prompt: str) -> Tuple[str, str]:
         animate_thinking("Thinking...", color="status")
         self.memory.push('user', prompt)
-        answer, reasoning = self.llm_request(prompt)
+        answer, reasoning = self.llm_request()
         pretty_print("-"*100)
         pretty_print(answer, color="output")
         pretty_print("-"*100)
         return answer, reasoning
     
-    def select_unvisited(self, search_result):
+    def select_unvisited(self, search_result: List[str]) -> List[str]:
         results_unvisited = []
         for res in search_result:
             if res["link"] not in self.search_history:
                 results_unvisited.append(res) 
         return results_unvisited
 
-    def jsonify_search_results(self, results_string):
+    def jsonify_search_results(self, results_string: str) -> List[str]:
         result_blocks = results_string.split("\n\n")
         parsed_results = []
         for block in result_blocks:
@@ -114,62 +167,96 @@ class BrowserAgent(Agent):
                 parsed_results.append(result_dict)
         return parsed_results 
     
-    def stringify_search_results(self, results_arr):
+    def stringify_search_results(self, results_arr: List[str]) -> str:
         return '\n\n'.join([f"Link: {res['link']}" for res in results_arr])
     
     def save_notes(self, text):
         lines = text.split('\n')
         for line in lines:
-            if "summary:" in line.lower():
+            if "note" in line.lower():
                 self.notes.append(line)
     
-    def conclude_prompt(self, user_query):
+    def conclude_prompt(self, user_query: str) -> str:
-        search_note = '\n -'.join(self.notes)
+        annotated_notes = [f"{i+1}: {note.lower().replace('note:', '')}" for i, note in enumerate(self.notes)]
+        search_note = '\n'.join(annotated_notes)
+        print("AI research notes:\n", search_note)
         return f"""
-        Following a web search about:
+        Following a human request:
         {user_query}
-        Write a conclusion based on these notes:
+        A web AI made the following finding across different pages:
         {search_note}
+
+        Summarize the finding or step that lead to success, and provide a conclusion that answer the request.
+        """
+    
+    def search_prompt(self, user_prompt: str) -> str:
+        return f"""
+        Current date: {self.date}
+        Make a efficient search engine query to help users with their request:
+        {user_prompt}
+        Example:
+        User: "go to twitter, login with username toto and password pass79 to my twitter and say hello everyone "
+        You: search: Twitter login page. 
+
+        User: "I need info on the best laptops for AI this year."
+        You: "search: best laptops 2025 to run Machine Learning model, reviews"
+
+        User: "Search for recent news about space missions."
+        You: "search: Recent space missions news, {self.date}"
+
+        Do not explain, do not write anything beside the search query.
         """
 
     def process(self, user_prompt, speech_module) -> str:
         complete = False
 
+        animate_thinking(f"Thinking...", color="status")
+        self.memory.push('user', self.search_prompt(user_prompt))
+        ai_prompt, _ = self.llm_request()
         animate_thinking(f"Searching...", color="status")
-        search_result_raw = self.tools["web_search"].execute([user_prompt], False)
+        search_result_raw = self.tools["web_search"].execute([ai_prompt], False)
-        search_result = self.jsonify_search_results(search_result_raw)
+        search_result = self.jsonify_search_results(search_result_raw)[:7] # until futher improvement
-        search_result = search_result[:10] # until futher improvement
         prompt = self.make_newsearch_prompt(user_prompt, search_result)
         unvisited = [None]
         while not complete:
             answer, reasoning = self.llm_decide(prompt)
             self.save_notes(answer)
+
+            extracted_form = self.extract_form(answer)
+            if len(extracted_form) > 0:
+                self.browser.fill_form_inputs(extracted_form)
+                self.browser.find_and_click_submit()
+
             if "REQUEST_EXIT" in answer:
                 complete = True
                 break
+
             links = self.extract_links(answer)
+            if len(unvisited) == 0:
+                break
+
             if len(links) == 0 or "GO_BACK" in answer:
                 unvisited = self.select_unvisited(search_result)
                 prompt = self.make_newsearch_prompt(user_prompt, unvisited)
                 pretty_print(f"Going back to results. Still {len(unvisited)}", color="warning")
                 links = []
                 continue
-            if len(unvisited) == 0:
+
-                break
             animate_thinking(f"Navigating to {links[0]}", color="status")
             speech_module.speak(f"Navigating to {links[0]}")
             self.browser.go_to(links[0])
+            self.current_page = links[0]
             self.search_history.append(links[0])
             page_text = self.browser.get_text()
             self.navigable_links = self.browser.get_navigable()
             prompt = self.make_navigation_prompt(user_prompt, page_text)
 
-        speech_module.speak(answer)
         self.browser.close()
         prompt = self.conclude_prompt(user_prompt)
-        answer, reasoning = self.llm_request(prompt)
+        self.memory.push('user', prompt)
+        answer, reasoning = self.llm_request()
         pretty_print(answer, color="output")
         return answer, reasoning
 
 if __name__ == "__main__":
-    browser = Browser()
+    pass

sources/agents/casual_agent.py

@@ -7,24 +7,24 @@ from sources.tools.fileFinder import FileFinder
 from sources.tools.BashInterpreter import BashInterpreter
 
 class CasualAgent(Agent):
-    def __init__(self, model, name, prompt_path, provider):
+    def __init__(self, name, prompt_path, provider, verbose=False):
         """
         The casual agent is a special for casual talk to the user without specific tasks.
         """
-        super().__init__(model, name, prompt_path, provider)
+        super().__init__(name, prompt_path, provider, verbose)
         self.tools = {
             "web_search": searxSearch(),
             "flight_search": FlightSearch(),
             "file_finder": FileFinder(),
             "bash": BashInterpreter()
         }
-        self.role = "casual talking"
+        self.role = "talk"
+        self.type = "casual_agent"
     
     def process(self, prompt, speech_module) -> str:
         complete = False
         self.memory.push('user', prompt)
 
-        self.wait_message(speech_module)
         while not complete:
             animate_thinking("Thinking...", color="status")
             answer, reasoning = self.llm_request()

sources/agents/code_agent.py

@@ -11,8 +11,8 @@ class CoderAgent(Agent):
     """
     The code agent is an agent that can write and execute code.
     """
-    def __init__(self, model, name, prompt_path, provider):
+    def __init__(self, name, prompt_path, provider, verbose=False):
-        super().__init__(model, name, prompt_path, provider)
+        super().__init__(name, prompt_path, provider, verbose)
         self.tools = {
             "bash": BashInterpreter(),
             "python": PyInterpreter(),
@@ -20,7 +20,8 @@ class CoderAgent(Agent):
             "go": GoInterpreter(),
             "file_finder": FileFinder()
         }
-        self.role = "coding and programming"
+        self.role = "Coding task"
+        self.type = "code_agent"
 
     def process(self, prompt, speech_module) -> str:
         answer = ""

sources/agents/file_agent.py

@@ -5,35 +5,28 @@ from sources.tools.fileFinder import FileFinder
 from sources.tools.BashInterpreter import BashInterpreter
 
 class FileAgent(Agent):
-    def __init__(self, model, name, prompt_path, provider):
+    def __init__(self, name, prompt_path, provider, verbose=False):
         """
         The file agent is a special agent for file operations.
         """
-        super().__init__(model, name, prompt_path, provider)
+        super().__init__(name, prompt_path, provider, verbose)
         self.tools = {
             "file_finder": FileFinder(),
             "bash": BashInterpreter()
         }
-        self.role = "files operations"
+        self.role = "find and read files"
+        self.type = "file_agent"
     
     def process(self, prompt, speech_module) -> str:
-        complete = False
         exec_success = False
         self.memory.push('user', prompt)
 
         self.wait_message(speech_module)
-        while not complete:
+        animate_thinking("Thinking...", color="status")
-            if exec_success:
+        answer, reasoning = self.llm_request()
-                complete = True
+        exec_success, _ = self.execute_modules(answer)
-            animate_thinking("Thinking...", color="status")
+        answer = self.remove_blocks(answer)
-            answer, reasoning = self.llm_request()
+        self.last_answer = answer
-            exec_success, _ = self.execute_modules(answer)
-            answer = self.remove_blocks(answer)
-            self.last_answer = answer
-            complete = True
-            for name, tool in self.tools.items():
-                if tool.found_executable_blocks():
-                    complete = False # AI read results and continue the conversation
         return answer, reasoning
 
 if __name__ == "__main__":

sources/agents/planner_agent.py

@@ -3,15 +3,15 @@ from sources.utility import pretty_print, animate_thinking
 from sources.agents.agent import Agent
 from sources.agents.code_agent import CoderAgent
 from sources.agents.file_agent import FileAgent
-from sources.agents.casual_agent import CasualAgent
+from sources.agents.browser_agent import BrowserAgent
 from sources.tools.tools import Tools
 
 class PlannerAgent(Agent):
-    def __init__(self, model, name, prompt_path, provider):
+    def __init__(self, name, prompt_path, provider, verbose=False):
         """
         The planner agent is a special agent that divides and conquers the task.
         """
-        super().__init__(model, name, prompt_path, provider)
+        super().__init__(name, prompt_path, provider, verbose)
         self.tools = {
             "json": Tools()
         }
@@ -19,10 +19,10 @@ class PlannerAgent(Agent):
         self.agents = {
             "coder": CoderAgent(model, name, prompt_path, provider),
             "file": FileAgent(model, name, prompt_path, provider),
-            "web": CasualAgent(model, name, prompt_path, provider)
+            "web": BrowserAgent(model, name, prompt_path, provider)
         }
-        self.role = "complex programming tasks and web research"
+        self.role = "Research, setup and code"
-        self.tag = "json"
+        self.type = "planner_agent"
 
     def parse_agent_tasks(self, text):
         tasks = []

sources/browser.py

@@ -5,11 +5,19 @@ from selenium.webdriver.common.by import By
 from selenium.webdriver.support.ui import WebDriverWait
 from selenium.webdriver.support import expected_conditions as EC
 from selenium.common.exceptions import TimeoutException, WebDriverException
+from selenium.webdriver.common.action_chains import ActionChains
+from selenium.webdriver.chrome.options import Options
+from typing import List, Tuple
+import chromedriver_autoinstaller
 import time
+import os
+import shutil
 from bs4 import BeautifulSoup
 import markdownify
 import logging
 import sys
+import re
+from urllib.parse import urlparse
 
 class Browser:
     def __init__(self, headless=False, anticaptcha_install=False):
@@ -20,46 +28,102 @@ class Browser:
             'Accept-Language': 'en-US,en;q=0.9',
             'Referer': 'https://www.google.com/',
         }
+        self.js_scripts_folder = "./sources/web_scripts/" if not __name__ == "__main__" else "./web_scripts/"
         self.anticaptcha = "https://chrome.google.com/webstore/detail/nopecha-captcha-solver/dknlfmjaanfblgfdfebhijalfmhmjjjo/related"
         try:
             chrome_options = Options()
+            chrome_path = self.get_chrome_path()
+            
+            if not chrome_path:
+                raise FileNotFoundError("Google Chrome not found. Please install it.")
+            chrome_options.binary_location = chrome_path
+            
             if headless:
                 chrome_options.add_argument("--headless")
             chrome_options.add_argument("--disable-gpu")
             chrome_options.add_argument("--no-sandbox")
             chrome_options.add_argument("--disable-dev-shm-usage")
-            self.driver = webdriver.Chrome(options=chrome_options)
+            chrome_options.add_argument("--autoplay-policy=user-gesture-required")
+            chrome_options.add_argument("--mute-audio")
+            chrome_options.add_argument("--disable-webgl")
+            chrome_options.add_argument("--disable-notifications")
+            security_prefs = {
+                "profile.default_content_setting_values.media_stream": 2,  # Block webcam/mic
+                "profile.default_content_setting_values.notifications": 2,  # Block notifications
+                "profile.default_content_setting_values.popups": 2,  # Block pop-ups
+                "profile.default_content_setting_values.geolocation": 2,  # Block geolocation
+                "safebrowsing.enabled": True,  # Enable safe browsing
+            }
+            chrome_options.add_experimental_option("prefs", security_prefs)
+            
+            chromedriver_path = shutil.which("chromedriver") # system installed driver.
+            
+            #If not found, try auto-installing the correct version
+            if not chromedriver_path:
+                chromedriver_path = chromedriver_autoinstaller.install()
+          
+            if not chromedriver_path:
+                raise FileNotFoundError("ChromeDriver not found. Please install it or add it to your PATH.")
+                
+            service = Service(chromedriver_path)
+            self.driver = webdriver.Chrome(service=service, options=chrome_options)
             self.wait = WebDriverWait(self.driver, 10)
             self.logger = logging.getLogger(__name__)
             self.logger.info("Browser initialized successfully")
         except Exception as e:
             raise Exception(f"Failed to initialize browser: {str(e)}")
+        self.load_anticatpcha()
             
-    def go_to(self, url):
+    @staticmethod
+    def get_chrome_path() -> str:
+        if sys.platform.startswith("win"):
+            paths = [
+                "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe",
+                "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe",
+                os.path.join(os.environ.get("LOCALAPPDATA", ""), "Google\\Chrome\\Application\\chrome.exe")  # User install
+            ]
+        elif sys.platform.startswith("darwin"):  # macOS
+            paths = ["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
+                     "/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"]
+        else:  # Linux
+            paths = ["/usr/bin/google-chrome", "/usr/bin/chromium-browser", "/usr/bin/chromium"]
+
+        for path in paths:
+            if os.path.exists(path) and os.access(path, os.X_OK):  # Check if executable
+                return path
+        return None
+    
+    def load_anticatpcha(self):
+        print("You might want to install the AntiCaptcha extension for captchas.")
+        self.driver.get(self.anticaptcha)
+
+    def go_to(self, url:str) -> bool:
         """Navigate to a specified URL."""
         try:
+            initial_handles = self.driver.window_handles
             self.driver.get(url)
-            time.sleep(2)  # Wait for page to load
+            time.sleep(1)
+            self.apply_web_safety()
             self.logger.info(f"Navigated to: {url}")
             return True
         except WebDriverException as e:
             self.logger.error(f"Error navigating to {url}: {str(e)}")
             return False
 
-    def is_sentence(self, text):
+    def is_sentence(self, text:str) -> bool:
         """Check if the text qualifies as a meaningful sentence or contains important error codes."""
         text = text.strip()
+
         error_codes = ["404", "403", "500", "502", "503"]
         if any(code in text for code in error_codes):
             return True
-        words = text.split()
+        words = re.findall(r'\w+', text, re.UNICODE)
         word_count = len(words)
-        has_punctuation = text.endswith(('.', '!', '?'))
+        has_punctuation = any(text.endswith(p) for p in ['.', '，', ',', '!', '?', '。', '！', '？', '।', '۔'])
         is_long_enough = word_count > 5
-        has_letters = any(word.isalpha() for word in words)
+        return (word_count >= 5 and (has_punctuation or is_long_enough))
-        return (word_count >= 5 and (has_punctuation or is_long_enough) and has_letters)
 
-    def get_text(self):
+    def get_text(self) -> str | None:
         """Get page text and convert it to README (Markdown) format."""
         try:
             soup = BeautifulSoup(self.driver.page_source, 'html.parser')
@@ -72,15 +136,14 @@ class Browser:
             lines = (line.strip() for line in text.splitlines())
             chunks = (phrase.strip() for line in lines for phrase in line.split("  "))
             text = "\n".join(chunk for chunk in chunks if chunk and self.is_sentence(chunk))
-            
+            #markdown_text = markdownify.markdownify(text, heading_style="ATX")
-            markdown_text = markdownify.markdownify(text, heading_style="ATX")
+            return "[Start of page]\n" + text + "\n[End of page]"
-            
-            return markdown_text
         except Exception as e:
             self.logger.error(f"Error getting text: {str(e)}")
             return None
     
-    def clean_url(self, url):
+    def clean_url(self, url:str) -> str:
+        """Clean URL to keep only the part needed for navigation to the page"""
         clean = url.split('#')[0]
         parts = clean.split('?', 1)
         base_url = parts[0]
@@ -96,7 +159,23 @@ class Browser:
                 return f"{base_url}?{'&'.join(essential_params)}"
         return base_url
     
-    def get_navigable(self):
+    def is_link_valid(self, url:str) -> bool:
+        """Check if a URL is a valid link (page, not related to icon or metadata)."""
+        if len(url) > 64:
+            return False
+        parsed_url = urlparse(url)
+        if not parsed_url.scheme or not parsed_url.netloc:
+            return False
+        if re.search(r'/\d+$', parsed_url.path):
+            return False
+        image_extensions = ['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp']
+        metadata_extensions = ['.ico', '.xml', '.json', '.rss', '.atom']
+        for ext in image_extensions + metadata_extensions:
+            if url.lower().endswith(ext):
+                return False
+        return True
+
+    def get_navigable(self) -> [str]:
         """Get all navigable links on the current page."""
         try:
             links = []
@@ -112,33 +191,149 @@ class Browser:
                     })
             
             self.logger.info(f"Found {len(links)} navigable links")
-            return [self.clean_url(link['url']) for link in links if link['is_displayed'] == True and len(link) < 256]
+            return [self.clean_url(link['url']) for link in links if (link['is_displayed'] == True and self.is_link_valid(link['url']))]
         except Exception as e:
             self.logger.error(f"Error getting navigable links: {str(e)}")
             return []
 
-    def click_element(self, xpath):
+    def click_element(self, xpath: str) -> bool:
-        """Click an element specified by xpath."""
+        """Click an element specified by XPath."""
         try:
-            element = self.wait.until(
+            element = self.wait.until(EC.element_to_be_clickable((By.XPATH, xpath)))
-                EC.element_to_be_clickable((By.XPATH, xpath))
+            if not element.is_displayed():
-            )
+                return False
-            element.click()
+            if not element.is_enabled():
-            time.sleep(2)  # Wait for action to complete
+                return False
-            return True
+            try:
+                self.driver.execute_script("arguments[0].scrollIntoView({block: 'center', behavior: 'smooth'});", element)
+                time.sleep(0.1)
+                element.click()
+                return True
+            except ElementClickInterceptedException as e:
+                return False
         except TimeoutException:
-            self.logger.error(f"Element not found or not clickable: {xpath}")
+            return False
+        except Exception as e:
+            self.logger.error(f"Unexpected error clicking element at {xpath}: {str(e)}")
             return False
         
-    def get_current_url(self):
+    def load_js(self, file_name: str) -> str:
+        path = os.path.join(self.js_scripts_folder, file_name)
+        try:
+            with open(path, 'r') as f:
+                return f.read()
+        except FileNotFoundError as e:
+            raise Exception(f"Could not find: {path}") from e
+        except Exception as e:
+            raise e
+
+    def find_all_inputs(self, timeout=4):
+        WebDriverWait(self.driver, timeout).until(
+            EC.presence_of_element_located((By.TAG_NAME, "body"))
+        )
+        time.sleep(0.5)
+        script = self.load_js("find_inputs.js")
+        input_elements = self.driver.execute_script(script)
+        return input_elements
+
+    def get_form_inputs(self) -> List[str]:
+        """Extract all input from the page and return them."""
+        try:
+            #input_elements = self.driver.find_elements(By.TAG_NAME, "input")
+            input_elements = self.find_all_inputs()
+            if not input_elements:
+                return ["No input forms found on the page."]
+
+            form_strings = []
+            for element in input_elements:
+                input_type = element["type"] or "text"
+                if input_type in ["hidden", "submit", "button", "image"] or not element["displayed"]:
+                    continue
+                input_name = element["text"] or element["id"] or input_type
+                if input_type == "checkbox" or input_type == "radio":
+                    checked_status = "checked" if element.is_selected() else "unchecked"
+                    form_strings.append(f"[{input_name}]({checked_status})")
+                else:
+                    form_strings.append(f"[{input_name}]("")")
+            return form_strings
+
+        except Exception as e:
+            self.logger.error(f"Error extracting form inputs: {str(e)}")
+            return [f"Error extracting form inputs."]
+
+    def get_buttons_xpath(self) -> List[str]:
+        """
+        Find buttons and return their type and xpath.
+        """
+        buttons = self.driver.find_elements(By.TAG_NAME, "button") + \
+                  self.driver.find_elements(By.XPATH, "//input[@type='submit']")
+        result = []
+        for i, button in enumerate(buttons):
+            if not button.is_displayed() or not button.is_enabled():
+                continue
+            text = (button.text or button.get_attribute("value") or "").lower().replace(' ', '')
+            xpath = f"(//button | //input[@type='submit'])[{i + 1}]"
+            if "login" in text or "sign" in text or "register":
+                result.append((text, xpath))
+        result.sort(key=lambda x: len(x[0]))
+        return result
+
+    def find_and_click_submit(self, btn_type:str = 'login') -> None:
+        buttons = self.get_buttons_xpath()
+        if len(buttons) == 0:
+            self.logger.warning(f"No visible buttons found")
+        for button in buttons:
+            if button[0] == btn_type:
+                self.click_element(button[1])
+    
+    def find_input_xpath_by_name(self, inputs, name: str) -> str | None:
+        for field in inputs:
+            if name in field["text"]:
+                return field["xpath"]
+        return None
+
+    def fill_form_inputs(self, input_list:[str]) -> bool:
+        """Fill form inputs based on a list of [name](value) strings."""
+        inputs = self.find_all_inputs()
+        try:
+            for input_str in input_list:
+                match = re.match(r'\[(.*?)\]\((.*?)\)', input_str)
+                if not match:
+                    self.logger.warning(f"Invalid format for input: {input_str}")
+                    continue
+
+                name, value = match.groups()
+                name = name.strip()
+                value = value.strip()
+                xpath = self.find_input_xpath_by_name(inputs, name)
+                if not xpath:
+                    continue
+                element = self.driver.find_element(By.XPATH, xpath)
+                input_type = (element.get_attribute("type") or "text").lower()
+                if input_type in ["checkbox", "radio"]:
+                    is_checked = element.is_selected()
+                    should_be_checked = value.lower() == "checked"
+
+                    if is_checked != should_be_checked:
+                        element.click()
+                        self.logger.info(f"Set {name} to {value}")
+                else:
+                    element.send_keys(value)
+                    self.logger.info(f"Filled {name} with {value}")
+            return True
+        except Exception as e:
+            self.logger.error(f"Error filling form inputs: {str(e)}")
+            return False
+
+    def get_current_url(self) -> str:
         """Get the current URL of the page."""
         return self.driver.current_url
 
-    def get_page_title(self):
+    def get_page_title(self) -> str:
         """Get the title of the current page."""
         return self.driver.title
 
-    def scroll_bottom(self):
+    def scroll_bottom(self) -> bool:
         """Scroll to the bottom of the page."""
         try:
             self.driver.execute_script(
@@ -150,7 +345,7 @@ class Browser:
             self.logger.error(f"Error scrolling: {str(e)}")
             return False
 
-    def screenshot(self, filename):
+    def screenshot(self, filename:str) -> bool:
         """Take a screenshot of the current page."""
         try:
             self.driver.save_screenshot(filename)
@@ -160,6 +355,13 @@ class Browser:
             self.logger.error(f"Error taking screenshot: {str(e)}")
             return False
 
+    def apply_web_safety(self):
+        """
+        Apply security measures to block any website malicious/annoying execution, privacy violation etc..
+        """
+        script = self.load_js("inject_safety_script.js")
+        input_elements = self.driver.execute_script(script)
+
     def close(self):
         """Close the browser."""
         try:
@@ -176,13 +378,19 @@ if __name__ == "__main__":
     logging.basicConfig(level=logging.INFO)
     
     browser = Browser(headless=False)
+    time.sleep(8)
     
     try:
-        browser.go_to("https://karpathy.github.io/")
+        print("AntiCaptcha Test")
-        text = browser.get_text()
+        browser.go_to("https://www.google.com/recaptcha/api2/demo")
-        print("Page Text in Markdown:")
+        time.sleep(5)
-        print(text)
+        print("Form Test:")
-        links = browser.get_navigable()
+        browser.go_to("https://practicetestautomation.com/practice-test-login/")
-        print("\nNavigable Links:", links)
+        inputs = browser.get_form_inputs()
+        inputs = ['[username](student)', f'[password](Password123)', '[appOtp]()', '[backupOtp]()']
+        browser.fill_form_inputs(inputs)
+        browser.find_and_click_submit()
+        print("Stress test")
+        browser.go_to("https://theannoyingsite.com/")
     finally:
         browser.close()

sources/interaction.py

@@ -27,7 +27,7 @@ class Interaction:
             self.transcriber = AudioTranscriber(self.ai_name, verbose=False)
             self.recorder = AudioRecorder()
         if tts_enabled:
-            self.speech.speak("Hello Sir, we are online and ready. What can I do for you ?")
+            self.speech.speak("Hello, we are online and ready. What can I do for you ?")
         if recover_last_session:
             self.recover_last_session()
     
@@ -35,7 +35,7 @@ class Interaction:
         """Find the name of the default AI. It is required for STT as a trigger word."""
         ai_name = "jarvis"
         for agent in self.agents:
-            if agent.role == "talking":
+            if agent.type == "casual_agent":
                 ai_name = agent.agent_name
                 break
         return ai_name
@@ -43,12 +43,12 @@ class Interaction:
     def recover_last_session(self):
         """Recover the last session."""
         for agent in self.agents:
-            agent.memory.load_memory()
+            agent.memory.load_memory(agent.type)
     
     def save_session(self):
         """Save the current session."""
         for agent in self.agents:
-            agent.memory.save_memory()
+            agent.memory.save_memory(agent.type)
 
     def is_active(self) -> bool:
         return self.is_active
@@ -58,7 +58,7 @@ class Interaction:
         buffer = ""
 
         PROMPT = "\033[1;35m➤➤➤ \033[0m"
-        while buffer == "" or buffer.isascii() == False:
+        while not buffer:
             try:
                 buffer = input(PROMPT)
             except EOFError:
@@ -98,10 +98,11 @@ class Interaction:
         agent = self.router.select_agent(self.last_query)
         if agent is None:
             return
-        if self.current_agent != agent:
+        if self.current_agent != agent and self.last_answer is not None:
-            self.current_agent = agent
+            ## get history from previous agent, good ?
-            # get history from previous agent, good ?
             self.current_agent.memory.push('user', self.last_query)
+            self.current_agent.memory.push('assistant', self.last_answer)
+        self.current_agent = agent
         self.last_answer, _ = agent.process(self.last_query, self.speech)
     
     def show_answer(self) -> None:

sources/language.py

@@ -0,0 +1,84 @@
+import langid
+import re
+import nltk
+from nltk.sentiment.vader import SentimentIntensityAnalyzer
+
+class LanguageUtility:
+    """LanguageUtility for language, or emotion identification"""
+    def __init__(self):
+        try:
+            nltk.data.find('vader_lexicon')
+        except LookupError:
+            nltk.download('vader_lexicon')
+        self.sid = SentimentIntensityAnalyzer()
+    
+    def detect_language(self, text: str) -> str:
+        """
+        Detect the language of the given text using langdetect
+        Args:
+            text: string to analyze
+        Returns: ISO639-1 language code
+        """
+        langid.set_languages(['fr', 'en', 'zh', 'es'])  # ISO 639-1 codes
+        lang, score = langid.classify(text)
+        return lang
+
+    def detect_emotion(self, text: str) -> str:
+        """
+        Detect the dominant emotion in the given text
+        Args:
+            text: string to analyze
+        Returns: string of the dominant emotion
+        """
+        try:
+            scores = self.sid.polarity_scores(text)
+            emotions = {
+                'Happy': max(scores['pos'], 0),
+                'Angry': 0,
+                'Sad': max(scores['neg'], 0),
+                'Fear': 0,
+                'Surprise': 0
+            }
+            if scores['compound'] < -0.5:
+                emotions['Angry'] = abs(scores['compound']) * 0.5
+                emotions['Fear'] = abs(scores['compound']) * 0.5
+            elif scores['compound'] > 0.5:
+                emotions['Happy'] = scores['compound']
+                emotions['Surprise'] = scores['compound'] * 0.5
+            dominant_emotion = max(emotions, key=emotions.get)
+            if emotions[dominant_emotion] == 0:
+                return 'Neutral'
+            return dominant_emotion
+        except Exception as e:
+            raise e
+    
+    def analyze(self, text):
+        """
+        Combined analysis of language and emotion
+        Args:
+            text: string to analyze
+        Returns: dictionary with language and emotion results
+        """
+        try:
+            language = self.detect_language(text)
+            emotions = self.detect_emotion(text)
+            return {
+                "language": language,
+                "emotions": emotions
+            }
+        except Exception as e:
+            raise e
+
+if __name__ == "__main__":
+    detector = LanguageUtility()
+    
+    test_texts = [
+        "I am so happy today!",
+        "Qué tristeza siento ahora",
+        "我不要去巴黎",
+        "La vie c'est cool"
+    ]
+    for text in test_texts:
+        print(f"\nAnalyzing: {text}")
+        result = detector.analyze(text)
+        print(result)

sources/llm_provider.py

@@ -12,6 +12,8 @@ from huggingface_hub import InferenceClient
 import os
 import httpx
 
+from sources.utility import pretty_print, animate_thinking
+
 class Provider:
     def __init__(self, provider_name, model, server_address = "127.0.0.1:5000"):
         self.provider_name = provider_name.lower()
@@ -21,17 +23,18 @@ class Provider:
             "ollama": self.ollama_fn,
             "server": self.server_fn,
             "openai": self.openai_fn,
-            "huggingface": self.huggingface_fn
+            "huggingface": self.huggingface_fn,
+            "deepseek-api": self.deepseek_fn
         }
         self.api_key = None
-        self.unsafe_providers = ["openai"]
+        self.unsafe_providers = ["openai", "deepseek-api"]
         if self.provider_name not in self.available_providers:
             raise ValueError(f"Unknown provider: {provider_name}")
         if self.provider_name in self.unsafe_providers:
-            print("Warning: you are using an API provider. You data will be sent to the cloud.")
+            pretty_print("Warning: you are using an API provider. You data will be sent to the cloud.", color="warning")
-            self.get_api_key(self.provider_name)
+            self.api_key = self.get_api_key(self.provider_name)
-        elif self.server != "":
+        elif self.server != "ollama":
-            print("Provider", provider_name, "initialized at", self.server)
+            pretty_print(f"Provider: {provider_name} initialized at {self.server}", color="success")
         self.check_address_format(self.server)
         if not self.is_ip_online(self.server.split(':')[0]):
             raise Exception(f"Server at {self.server} is offline.")
@@ -51,6 +54,7 @@ class Provider:
         Validate if the address is valid IP.
         """
         try:
+            address = address.replace('http://', '')
             ip, port = address.rsplit(":", 1)
             if all(c.lower() in ".:abcdef0123456789" for c in ip):
                 ipaddress.ip_address(ip)
@@ -79,6 +83,8 @@ class Provider:
         """
         Check if an IP address is online by sending a ping request.
         """
+        if ip_address == "127.0.0.1":
+            return True
         param = '-n' if platform.system().lower() == 'windows' else '-c'
         command = ['ping', param, '1', ip_address]
         try:
@@ -86,13 +92,11 @@ class Provider:
             if output.returncode == 0:
                 return True
             else:
-                print("errorcode:", output)
                 return False
         except subprocess.TimeoutExpired:
-            print("timeout")
+            return False
-            return True
         except Exception as e:
-            print(f"is_ip_online error:\n{e}")
+            pretty_print(f"Error with ping request {str(e)}", color="failure")
             return False
 
     def server_fn(self, history, verbose = False):
@@ -114,7 +118,7 @@ class Provider:
                 is_complete = bool(response.json()["is_complete"])
                 time.sleep(2)
         except KeyError as e:
-            raise f"{str(e)}\n\nError occured with server route. Are you using the correct address for the config.ini provider?"
+            raise Exception(f"{str(e)}\n\nError occured with server route. Are you using the correct address for the config.ini provider?") from e
         except Exception as e:
             raise e
         return thought
@@ -138,8 +142,9 @@ class Provider:
             raise Exception("\nOllama connection failed. provider should not be set to ollama if server address is not localhost") from e
         except ollama.ResponseError as e:
             if e.status_code == 404:
-                print(f"Downloading {self.model}...")
+                animate_thinking(f"Downloading {self.model}...")
                 ollama.pull(self.model)
+                self.ollama_fn(history, verbose)
             if "refused" in str(e).lower():
                 raise Exception("Ollama connection failed. is the server running ?") from e
             raise e
@@ -164,8 +169,7 @@ class Provider:
         """
         Use openai to generate text.
         """
-        api_key = self.get_api_key("openai")
+        client = OpenAI(api_key=self.api_key)
-        client = OpenAI(api_key=api_key)
         try:
             response = client.chat.completions.create(
                 model=self.model,
@@ -178,6 +182,24 @@ class Provider:
         except Exception as e:
             raise Exception(f"OpenAI API error: {str(e)}") from e
 
+    def deepseek_fn(self, history, verbose=False):
+        """
+        Use deepseek api to generate text.
+        """
+        client = OpenAI(api_key=self.api_key, base_url="https://api.deepseek.com")
+        try:
+            response = client.chat.completions.create(
+                model="deepseek-chat",
+                messages=history,
+                stream=False
+            )
+            thought = response.choices[0].message.content
+            if verbose:
+                print(thought)
+            return thought
+        except Exception as e:
+            raise Exception(f"Deepseek API error: {str(e)}") from e
+
     def test_fn(self, history, verbose = True):
         """
         This function is used to conduct tests.

sources/memory.py

@@ -20,7 +20,7 @@ class Memory():
                  recover_last_session: bool = False,
                  memory_compression: bool = True):
         self.memory = []
-        self.memory = [{'role': 'user', 'content': system_prompt}]
+        self.memory = [{'role': 'system', 'content': system_prompt}]
         
         self.session_time = datetime.datetime.now()
         self.session_id = str(uuid.uuid4())
@@ -38,20 +38,23 @@ class Memory():
     def get_filename(self) -> str:
         return f"memory_{self.session_time.strftime('%Y-%m-%d_%H-%M-%S')}.txt"
     
-    def save_memory(self) -> None:
+    def save_memory(self, agent_type: str = "casual_agent") -> None:
         """Save the session memory to a file."""
         if not os.path.exists(self.conversation_folder):
             os.makedirs(self.conversation_folder)
+        save_path = os.path.join(self.conversation_folder, agent_type)
+        if not os.path.exists(save_path):
+            os.makedirs(save_path)
         filename = self.get_filename()
-        path = os.path.join(self.conversation_folder, filename)
+        path = os.path.join(save_path, filename)
         json_memory = json.dumps(self.memory)
         with open(path, 'w') as f:
             f.write(json_memory)
     
-    def find_last_session_path(self) -> str:
+    def find_last_session_path(self, path) -> str:
         """Find the last session path."""
         saved_sessions = []
-        for filename in os.listdir(self.conversation_folder):
+        for filename in os.listdir(path):
             if filename.startswith('memory_'):
                 date = filename.split('_')[1]
                 saved_sessions.append((filename, date))
@@ -60,14 +63,15 @@ class Memory():
             return saved_sessions[0][0]
         return None
 
-    def load_memory(self) -> None:
+    def load_memory(self, agent_type: str = "casual_agent") -> None:
         """Load the memory from the last session."""
-        if not os.path.exists(self.conversation_folder):
+        save_path = os.path.join(self.conversation_folder, agent_type)
+        if not os.path.exists(save_path):
             return
-        filename = self.find_last_session_path()
+        filename = self.find_last_session_path(save_path)
         if filename is None:
             return
-        path = os.path.join(self.conversation_folder, filename)
+        path = os.path.join(save_path, filename)
         with open(path, 'r') as f:
             self.memory = json.load(f)
     
@@ -76,10 +80,10 @@ class Memory():
     
     def push(self, role: str, content: str) -> None:
         """Push a message to the memory."""
-        self.memory.append({'role': role, 'content': content})
-        # EXPERIMENTAL
         if self.memory_compression and role == 'assistant':
             self.compress()
+        # we don't compress the last message
+        self.memory.append({'role': role, 'content': content})
     
     def clear(self) -> None:
         self.memory = []
@@ -129,9 +133,9 @@ class Memory():
         if not self.memory_compression:
             return
         for i in range(len(self.memory)):
-            if i <= 2:
+            if i < 3:
                 continue
-            if self.memory[i]['role'] == 'assistant':
+            if len(self.memory[i]['content']) > 1024:
                 self.memory[i]['content'] = self.summarize(self.memory[i]['content'])
 
 if __name__ == "__main__":

sources/router.py

@@ -9,6 +9,7 @@ from sources.agents.agent import Agent
 from sources.agents.code_agent import CoderAgent
 from sources.agents.casual_agent import CasualAgent
 from sources.agents.planner_agent import PlannerAgent
+from sources.agents.browser_agent import BrowserAgent
 from sources.utility import pretty_print
 
 class AgentRouter:
@@ -30,7 +31,7 @@ class AgentRouter:
         else:
             return "cpu"
 
-    def classify_text(self, text: str, threshold: float = 0.5) -> list:
+    def classify_text(self, text: str, threshold: float = 0.4) -> list:
         """
         Classify the text into labels (agent roles).
         Args:
@@ -41,8 +42,8 @@ class AgentRouter:
         """
         first_sentence = None
         for line in text.split("\n"):
-                first_sentence = line.strip()
+            first_sentence = line.strip()
-                break
+            break
         if first_sentence is None:
             first_sentence = text
         result = self.pipeline(first_sentence, self.labels, threshold=threshold)
@@ -61,36 +62,56 @@ class AgentRouter:
         result = self.classify_text(text)
         for agent in self.agents:
             if result["labels"][0] == agent.role:
-                pretty_print(f"Selected agent: {agent.agent_name}", color="warning")
+                pretty_print(f"Selected agent: {agent.agent_name} (roles: {agent.role})", color="warning")
                 return agent
         return None
 
 if __name__ == "__main__":
     agents = [
-        CoderAgent("deepseek-r1:14b", "agent1", "../prompts/coder_agent.txt", "server"),
+        CasualAgent("deepseek-r1:14b", "jarvis", "../prompts/casual_agent.txt", None),
-        CasualAgent("deepseek-r1:14b", "agent2", "../prompts/casual_agent.txt", "server"),
+        BrowserAgent("deepseek-r1:14b", "browser", "../prompts/planner_agent.txt", None),
-        PlannerAgent("deepseek-r1:14b", "agent3", "../prompts/planner_agent.txt", "server")
+        CoderAgent("deepseek-r1:14b", "coder", "../prompts/coder_agent.txt", None)
     ]
     router = AgentRouter(agents)
-    
+    texts = [
-    texts = ["""
+        "Write a python script to check if the device on my network is connected to the internet",
-    Write a python script to check if the device on my network is connected to the internet
+        #"Peut tu écrire un script python qui vérifie si l'appareil sur mon réseau est connecté à internet?",
-    """,
+        #"写一个Python脚本，检查我网络上的设备是否连接到互联网",
-    """
+        "Hey could you search the web for the latest news on the tesla stock market ?",
-    Hey could you search the web for the latest news on the stock market ?
+        #"嘿，你能搜索网页上关于股票市场的最新新闻吗？",
-    """,
+        #"Yo, cherche sur internet comment va tesla en bourse.",
-    """
+        "I would like you to search for weather api and then make an app using this API",
-    hey can you give give a list of the files in the current directory ?
+        #"我想让你搜索天气API，然后用这个API做一个应用程序",
-    """,
+        #"J'aimerais que tu cherche une api météo et que l'utilise pour faire une application",
-    """
+        "Plan a 3-day trip to New York, including flights and hotels.",
-    Make a cool game to illustrate the current relation between USA and europe
+        #"计划一次为期3天的纽约之旅，包括机票和酒店。",
-    """
+        #"Planifie un trip de 3 jours à Paris, y compris les vols et hotels.",
+        "Find me the latest research papers on AI.",
+        #"给我找最新的AI研究论文。",
+        #"Trouve moi les derniers papiers de recherche en IA",
+        "Help me write a C++ program to sort an array",
+        #"帮我写一个C++程序来排序数组",
+        #"Aide moi à faire un programme c++ pour trier une array.",
+        "What’s the weather like today? Oh, and can you find a good weather app?",
+        #"今天天气怎么样？哦，你还能找到一个好的天气应用程序吗？",
+        #"La météo est comment aujourd'hui ? oh et trouve moi une bonne appli météo tant que tu y est.",
+        "Can you debug this Java code? It’s not working.",
+        #"你能调试这段Java代码吗？它不起作用。",
+        #"Peut tu m'aider à debugger ce code java, ça marche pas",
+        "What's the latest brainrot on the internet ?",
+        #"互联网上最新的“脑残”是什么？",
+        #"Quel est la dernière connerie sur internet ?",
+        "i would like to setup a new AI project, index as mark2",
+        #"我想建立一个新的 AI 项目，索引为 Mark2",
+        #"Je voudrais configurer un nouveau projet d'IA, index Mark2",
+        "Hey, can you find the old_project.zip file somewhere on my drive?",
+        #"嘿，你能在我驱动器上找到old_project.zip文件吗？",
+        #"Hé trouve moi le old_project.zip, il est quelque part sur mon disque.",
+        "Tell me a funny story",
+        #"给我讲一个有趣的故事",
+        #"Raconte moi une histoire drole"
     ]
-
     for text in texts:
-        print(text)
+        print("Input text:", text)
-        results = router.classify_text(text)
-        for result in results:
-            print(result["label"], "=>", result["score"])
         agent = router.select_agent(text)
-        print("Selected agent role:", agent.role)
+        print()

sources/speech_to_text.py

@@ -3,9 +3,10 @@ import queue
 import threading
 import numpy as np
 import torch
-from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
 import time
+from transformers import AutoModelForSpeechSeq2Seq, AutoProcessor, pipeline
 import librosa
+import pyaudio
 
 audio_queue = queue.Queue()
 done = False
@@ -14,8 +15,7 @@ class AudioRecorder:
     """
     AudioRecorder is a class that records audio from the microphone and adds it to the audio queue.
     """
-    def __init__(self, format: int, channels: int = 1, rate: int = 4096, chunk: int = 8192, record_seconds: int = 5, verbose: bool = False):
+    def __init__(self, format: int = pyaudio.paInt16, channels: int = 1, rate: int = 4096, chunk: int = 8192, record_seconds: int = 5, verbose: bool = False):
-        import pyaudio
         self.format = format
         self.channels = channels
         self.rate = rate

sources/text_to_speech.py

@@ -4,6 +4,7 @@ import soundfile as sf
 import subprocess
 import re
 import platform
+from sys import modules
 
 class Speech():
     """
@@ -24,7 +25,7 @@ class Speech():
         self.voice = self.voice_map[language][2]
         self.speed = 1.2
 
-    def speak(self, sentence: str, voice_number: int = 1):
+    def speak(self, sentence: str, voice_number: int = 1 , audio_file: str = 'sample.wav'):
         """
         Convert text to speech using an AI model and play the audio.
 
@@ -38,17 +39,17 @@ class Speech():
             sentence, voice=self.voice,
             speed=self.speed, split_pattern=r'\n+'
         )
-        for i, (gs, ps, audio) in enumerate(generator):
+        for i, (_, _, audio) in enumerate(generator):
-            audio_file = 'sample.wav'
+            if 'ipykernel' in modules: #only display in jupyter notebook.
-            display(Audio(data=audio, rate=24000, autoplay=i==0), display_id=False)
+                display(Audio(data=audio, rate=24000, autoplay=i==0), display_id=False)
             sf.write(audio_file, audio, 24000) # save each audio file
             if platform.system().lower() == "windows":
                 import winsound
                 winsound.PlaySound(audio_file, winsound.SND_FILENAME)
-            elif platform.system().lower() == "linux":
+            elif platform.system().lower() == "darwin":  # macOS
-                subprocess.call(["aplay", audio_file])
-            else:
                 subprocess.call(["afplay", audio_file])
+            else: # linux or other.
+                subprocess.call(["aplay", audio_file])
 
     def replace_url(self, url: re.Match) -> str:
         """

sources/tools/PyInterpreter.py

@@ -35,6 +35,7 @@ class PyInterpreter(Tools):
         try:
             try:
                 buffer = exec(code, global_vars)
+                print(buffer)
                 if buffer is not None:
                     output = buffer + '\n'
             except Exception as e:

sources/tools/searxSearch.py

@@ -50,7 +50,6 @@ class searxSearch(Tools):
         """Check all links, one by one."""
         # TODO Make it asyncromous or smth
         statuses = []
-        print("Web scrawl to verify links accessibilty...")
         for i, link in enumerate(links):
             status = self.link_valid(link)
             statuses.append(status)
@@ -76,7 +75,7 @@ class searxSearch(Tools):
             'Upgrade-Insecure-Requests': '1',
             'User-Agent': self.user_agent
         }
-        data = f"q={query}&categories=general&language=auto&time_range=&safesearch=0&theme=simple"
+        data = f"q={query}&categories=general&language=auto&time_range=&safesearch=0&theme=simple".encode('utf-8')
         try:
             response = requests.post(search_url, headers=headers, data=data, verify=False)
             response.raise_for_status()
@@ -90,9 +89,11 @@ class searxSearch(Tools):
                     title = article.find('h3').text.strip() if article.find('h3') else "No Title"
                     description = article.find('p', class_='content').text.strip() if article.find('p', class_='content') else "No Description"
                     results.append(f"Title:{title}\nSnippet:{description}\nLink:{url}")
+            if len(results) == 0:
+                return "No search results, web search failed."
             return "\n\n".join(results)  # Return results as a single string, separated by newlines
         except requests.exceptions.RequestException as e:
-            return f"Error during search: {str(e)}"
+            raise Exception("\nSearxng search failed. did you run start_services.sh? is docker still running?") from e
 
     def execution_failure_check(self, output: str) -> bool:
         """

sources/tools/tools.py

@@ -118,10 +118,8 @@ class Tools():
         save_path_file = os.path.basename(save_path)
         directory = os.path.join(self.current_dir, save_path_dir)
         if directory and not os.path.exists(directory):
-            print(f"Creating directory: {directory}")
             os.makedirs(directory)
         for block in blocks:
-            print(f"Saving code block to: {save_path}")
             with open(os.path.join(directory, save_path_file), 'w') as f:
                 f.write(block)
     

sources/tools/webSearch.py

@@ -58,7 +58,6 @@ class webSearch(Tools):
         """Check all links, one by one."""
         # TODO Make it asyncromous or smth
         statuses = []
-        print("Workers started, scrawling the web...")
         for i, link in enumerate(links):
             status = self.link_valid(link)
             statuses.append(status)

sources/web_scripts/find_inputs.js

@@ -0,0 +1,49 @@
+function findInputs(element, result = []) {
+    // Find all <input> elements in the current DOM tree
+    const inputs = element.querySelectorAll('input');
+    inputs.forEach(input => {
+        result.push({
+            tagName: input.tagName,
+            text: input.name || '',
+            type: input.type || '',
+            class: input.className || '',
+            xpath: getXPath(input),
+            displayed: isElementDisplayed(input)
+        });
+    });
+    const allElements = element.querySelectorAll('*');
+    allElements.forEach(el => {
+        if (el.shadowRoot) {
+            findInputs(el.shadowRoot, result);
+        }
+    });
+    return result;
+}
+// function to get the XPath of an element
+function getXPath(element) {
+    if (!element) return '';
+    if (element.id !== '') return '//*[@id="' + element.id + '"]';
+    if (element === document.body) return '/html/body';
+
+    let ix = 0;
+    const siblings = element.parentNode ? element.parentNode.childNodes : [];
+    for (let i = 0; i < siblings.length; i++) {
+        const sibling = siblings[i];
+        if (sibling === element) {
+            return getXPath(element.parentNode) + '/' + element.tagName.toLowerCase() + '[' + (ix + 1) + ']';
+        }
+        if (sibling.nodeType === 1 && sibling.tagName === element.tagName) {
+            ix++;
+        }
+    }
+    return '';
+}
+return findInputs(document.body);
+
+function isElementDisplayed(element) {
+    const style = window.getComputedStyle(element);
+    if (style.display === 'none' || style.visibility === 'hidden' || style.opacity === '0') {
+        return false;
+    }
+    return true;
+}

sources/web_scripts/inject_safety_script.js

@@ -0,0 +1,36 @@
+// Block hardware access by removing or disabling APIs
+Object.defineProperty(navigator, 'serial', { get: () => undefined });
+Object.defineProperty(navigator, 'hid', { get: () => undefined });
+Object.defineProperty(navigator, 'bluetooth', { get: () => undefined });
+// Block media playback
+HTMLMediaElement.prototype.play = function() {
+    this.pause(); // Immediately pause if play is called
+    return Promise.reject('Blocked by script');
+};
+// Block fullscreen requests
+Element.prototype.requestFullscreen = function() {
+    console.log('Blocked fullscreen request');
+    return Promise.reject('Blocked by script');
+};
+// Block pointer lock
+Element.prototype.requestPointerLock = function() {
+    console.log('Blocked pointer lock');
+};
+// Block iframe creation (optional, since browser already blocks these)
+const originalCreateElement = document.createElement;
+document.createElement = function(tagName) {
+    if (tagName.toLowerCase() === 'iframe') {
+        console.log('Blocked iframe creation');
+        return null;
+    }
+    return originalCreateElement.apply(this, arguments);
+};
+//block fetch
+window.fetch = function() {
+    console.log('Blocked fetch request');
+    return Promise.reject('Blocked');
+};
+// Block annoying dialogs
+window.alert = function() {};
+window.confirm = function() { return false; };
+window.prompt = function() { return null; };