From 63cd5eddd7d26cbc8b443432338fb755baecf208 Mon Sep 17 00:00:00 2001 From: Further <55025025+ifurther@users.noreply.github.com> Date: Mon, 26 May 2025 20:48:22 +0800 Subject: [PATCH 01/14] Update README_CHT.md --- README_CHT.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README_CHT.md b/README_CHT.md index fd0f301..2950708 100644 --- a/README_CHT.md +++ b/README_CHT.md @@ -8,15 +8,15 @@ [English](./README.md) | 繁體中文 | [日本語](./README_JP.md) -*一个 **100% 本地替代 Manus AI** 的方案，这款支持语音的 AI 助理能够自主浏览网页、编写代码和规划任务，同时将所有数据保留在您的设备上。专为本地推理模型量身打造，完全在您自己的硬件上运行，确保完全的隐私保护和零云端依赖。* +*一个 **100% 本地替代 Manus AI** 的方案，這款支持语音的 AI 助理能够自主瀏覽網頁、编写代码和規劃任務，同时将所有用戶資料保留在您的裝置上。專門為本地推理模型量身打造，完全在您自己的硬體上執行，确保完全的隐私保护和零雲端依賴。* [![Visit AgenticSeek](https://img.shields.io/static/v1?label=Website&message=AgenticSeek&color=blue&style=flat-square)](https://fosowl.github.io/agenticSeek.html) ![License](https://img.shields.io/badge/license-GPL--3.0-green) [![Discord](https://img.shields.io/badge/Discord-Join%20Us-7289DA?logo=discord&logoColor=white)](https://discord.gg/8hGDaME3TC) [![Twitter](https://img.shields.io/twitter/url/https/twitter.com/fosowl.svg?style=social&label=Update%20%40Fosowl)](https://x.com/Martin993886460) -### 为什么选择 AgenticSeek？ +### 为什么選擇 AgenticSeek？ * 🔒 完全本地化与隐私保护 - 所有功能都在您的设备上运行 — 无云端服务，无数据共享。您的文件、对话和搜索始终保持私密。 -* 🌐 智能网页浏览 - AgenticSeek 能够自主浏览互联网 — 搜索、阅读、提取信息、填写网页表单 — 全程无需人工操作。 +* 🌐 智能網頁浏览 - AgenticSeek 能够自主瀏覽網頁 — 搜索、阅读、提取信息、填写网页表单 — 全程无需人工操作。 * 💻 自主编码助手 - 需要代码？它可以编写、调试并运行 Python、C、Go、Java 等多种语言的程序 — 全程无需监督。 @@ -560,4 +560,4 @@ DeepSeek R1 天生会说中文 > [https://github.com/antoineVIVIES](https://github.com/antoineVIVIES) | 台北时间 | （经常很忙） - > [steveh8758](https://github.com/steveh8758) | 台北时间 | （总是很忙） \ No newline at end of file + > [steveh8758](https://github.com/steveh8758) | 台北时间 | （总是很忙） From 7ad084b27f7b1da7aafe3f62898592aed587bd02 Mon Sep 17 00:00:00 2001 From: ifurther <55025025+ifurther@users.noreply.github.com> Date: Mon, 26 May 2025 21:20:17 +0800 Subject: [PATCH 02/14] updat readme --- README_CHT.md | 92 +++++++++++++++++++++++++-------------------------- 1 file changed, 46 insertions(+), 46 deletions(-) diff --git a/README_CHT.md b/README_CHT.md index 2950708..eaf37bf 100644 --- a/README_CHT.md +++ b/README_CHT.md @@ -8,23 +8,23 @@ [English](./README.md) | 繁體中文 | [日本語](./README_JP.md) -*一个 **100% 本地替代 Manus AI** 的方案，這款支持语音的 AI 助理能够自主瀏覽網頁、编写代码和規劃任務，同时将所有用戶資料保留在您的裝置上。專門為本地推理模型量身打造，完全在您自己的硬體上執行，确保完全的隐私保护和零雲端依賴。* +*一个 **100% 本地替代 Manus AI** 的方案，這款支持語音的 AI 助理能够自主瀏覽網頁、编寫代码和規劃任務，同时將所有用戶資料保留在您的裝置上。專門為本地推理模型量身打造，完全在您自己的硬體上執行，确保完全的隐私保护和零雲端依賴。* [![Visit AgenticSeek](https://img.shields.io/static/v1?label=Website&message=AgenticSeek&color=blue&style=flat-square)](https://fosowl.github.io/agenticSeek.html) ![License](https://img.shields.io/badge/license-GPL--3.0-green) [![Discord](https://img.shields.io/badge/Discord-Join%20Us-7289DA?logo=discord&logoColor=white)](https://discord.gg/8hGDaME3TC) [![Twitter](https://img.shields.io/twitter/url/https/twitter.com/fosowl.svg?style=social&label=Update%20%40Fosowl)](https://x.com/Martin993886460) ### 为什么選擇 AgenticSeek？ -* 🔒 完全本地化与隐私保护 - 所有功能都在您的设备上运行 — 无云端服务，无数据共享。您的文件、对话和搜索始终保持私密。 +* 🔒 完全本地化與隐私保护 - 所有功能都在您的设备上運行 — 无云端服务，无数据共享。您的文件、对话和搜索始终保持私密。 -* 🌐 智能網頁浏览 - AgenticSeek 能够自主瀏覽網頁 — 搜索、阅读、提取信息、填写网页表单 — 全程无需人工操作。 +* 🌐 智能網頁瀏覽 - AgenticSeek 能够自主瀏覽網頁 — 搜索、閱读、提取信息、填寫網页表單 — 全程无需人工操作。 -* 💻 自主编码助手 - 需要代码？它可以编写、调试并运行 Python、C、Go、Java 等多种语言的程序 — 全程无需监督。 +* 💻 自主编码助手 - 需要代码？它可以编寫、调试并運行 Python、C、Go、Java 等多种语言的程序 — 全程无需监督。 -* 🧠 智能代理选择 - 您提问，它会自动选择最适合该任务的代理。就像拥有一个随时待命的专家团队。 +* 🧠 智能代理选择 - 您提问，它會自动选择最适合该任务的代理。就像拥有一个随时待命的專家团队。 -* 📋 规划与执行复杂任务 - 从旅行规划到复杂项目 — 它能将大型任务分解为步骤，并利用多个 AI 代理完成工作。 +* 📋 规划與执行复杂任务 - 从旅行规划到复杂项目 — 它能將大型任务分解为步骤，并利用多个 AI 代理完成工作。 -* 🎙️ 语音功能 - 清晰、快速、未来感十足的语音与语音转文本功能，让您能像科幻电影中一样与您的个人 AI 助手对话。 +* 🎙️ 語音功能 - 清晰、快速、未来感十足的語音與語音轉文本功能，讓您能像科幻电影中一样與您的个人 AI 助手对话。 https://github.com/user-attachments/assets/4bd5faf6-459f-4f94-bd1d-238c4b331469 @@ -36,7 +36,7 @@ https://github.com/user-attachments/assets/4bd5faf6-459f-4f94-bd1d-238c4b331469 確保已安裝了 Chrome driver，Docker 和 Python 3.10（或更新）。 -我们强烈建议您使用 Python 3.10 进行设置，否则可能会发生依赖错误。 +我们强烈建议您使用 Python 3.10 進行設定，否则可能會发生依赖错误。 有關於 Chrome driver 的問題，請參見 **Chromedriver** 部分。 @@ -64,7 +64,7 @@ source agentic_seek_env/bin/activate ./install.sh ``` -** 若要让文本转语音（TTS）功能支持中文，你需要安装 jieba（中文分词库）和 cn2an（中文数字转换库）：** +** 若要將文字轉成語音（TTS）功能支持中文，你需要安装 jieba（中文分詞庫）和 cn2an（中文數字轉換庫）：** ``` pip3 install jieba cn2an @@ -73,7 +73,7 @@ pip3 install jieba cn2an **手動安裝:** -**注意：对于任何操作系统，请确保您安装的 ChromeDriver 与您已安装的 Chrome 版本匹配。运行 `google-chrome --version`。如果您的 Chrome 版本 > 135，请参阅已知问题** +**注意：對於不同作業系統，請確保已經安装的 ChromeDriver 與您已安装的 Chrome 版本一致。可以執行 `google-chrome --version`。如果您的 Chrome 版本 > 135，請參考已知问题** - *Linux*: @@ -81,7 +81,7 @@ pip3 install jieba cn2an 安装依赖项：`sudo apt install -y alsa-utils portaudio19-dev python3-pyaudio libgtk-3-dev libnotify-dev libgconf-2-4 libnss3 libxss1` -安装与您的 Chrome 浏览器版本匹配的 ChromeDriver： +安装與您的 Chrome 瀏覽器版本匹配的 ChromeDriver： `sudo apt install -y chromium-chromedriver` 安装 requirements：`pip3 install -r requirements.txt` @@ -104,11 +104,11 @@ pip3 install jieba cn2an 安装 pyreadline3：`pip install pyreadline3` -手动安装 portaudio（例如，通过 vcpkg 或预编译的二进制文件），然后运行：`pip install pyaudio` +手动安装 portaudio（例如，通过 vcpkg 或預編譯的二進制文件），然後運行：`pip install pyaudio` -从以下网址手动下载并安装 chromedriver：https://sites.google.com/chromium.org/driver/getting-started +从以下網址手动下载并安装 chromedriver：https://sites.google.com/chromium.org/driver/getting-started -将 chromedriver 放置在包含在您的 PATH 中的目录中。 +將 chromedriver 放置在包含在您的 PATH 中的目录中。 安装 requirements：`pip3 install -r requirements.txt` @@ -116,45 +116,45 @@ pip3 install jieba cn2an **建議至少使用 Deepseek 14B 以上參數的模型，較小的模型難以使用助理功能並且很快就會忘記上下文之間的關係。** -**本地运行助手** +**本地運行助手** -启动你的本地提供者，例如使用 ollama： +啟動你的本地提供者，例如使用 ollama： ```sh ollama serve ``` -请参阅下方支持的本地提供者列表。 +请参閱下方支持的本地提供者列表。 -修改 config.ini 文件以设置 provider_name 为支持的提供者，并将 provider_model 设置为该提供者支持的 LLM。我们推荐使用具有推理能力的模型，如 *Qwen* 或 *Deepseek*。 +修改 config.ini 文件以設定 provider_name 为支持的提供者，并將 provider_model 設定为该提供者支持的 LLM。我们推荐使用具有推理能力的模型，如 *Qwen* 或 *Deepseek*。 请参见 README 末尾的 **FAQ** 部分了解所需硬件。 ```sh [MAIN] -is_local = True # 无论是在本地运行还是使用远程提供者。 +is_local = True # 无论是在本地運行还是使用远程提供者。 provider_name = ollama # 或 lm-studio, openai 等.. provider_model = deepseek-r1:14b # 选择适合您硬件的模型 provider_server_address = 127.0.0.1:11434 agent_name = Jarvis # 您的 AI 助手的名称 -recover_last_session = True # 是否恢复之前的会话 -save_session = True # 是否记住当前会话 -speak = True # 文本转语音 -listen = False # 语音转文本，仅适用于命令行界面 +recover_last_session = True # 是否恢复之前的會话 +save_session = True # 是否记住当前會话 +speak = True # 文本轉語音 +listen = False # 語音轉文本，僅适用于命令行界面 work_dir = /Users/mlg/Documents/workspace # AgenticSeek 的工作空间。 jarvis_personality = False # 是否使用更"贾维斯"风格的性格，不推荐在小型模型上使用 -languages = en zh # 语言列表，文本转语音将默认使用列表中的第一种语言 +languages = en zh # 语言列表，文本轉語音將默认使用列表中的第一种语言 [BROWSER] -headless_browser = True # 是否使用无头浏览器，只有在使用网页界面时才推荐使用。 -stealth_mode = True # 使用无法检测的 selenium 来减少浏览器检测 +headless_browser = True # 是否使用无头瀏覽器，只有在使用網页界面时才推荐使用。 +stealth_mode = True # 使用无法檢測的 selenium 来减少瀏覽器檢測 ``` **本地提供者列表** | 提供者 | 本地? | 描述 | |-------------|--------|-------------------------------------------------------| -| ollama | 是 | 使用 ollama 作为 LLM 提供者，轻松本地运行 LLM | -| lm-studio | 是 | 使用 LM Studio 本地运行 LLM（将 `provider_name` 设置为 `lm-studio`）| +| ollama | 是 | 使用 ollama 作为 LLM 提供者，轻松本地運行 LLM | +| lm-studio | 是 | 使用 LM Studio 本地運行 LLM（將 `provider_name` 設定为 `lm-studio`）| | openai | 否 | 使用兼容的 API | 下一步： [Start services and run AgenticSeek](#Start-services-and-Run) @@ -184,14 +184,14 @@ provider_server_address = 127.0.0.1:5000 --- ## Start services and Run -(启动服务并运行) +(啟動服务并運行) 如果需要，请激活你的 Python 环境。 ```sh source agentic_seek_env/bin/activate ``` -启动所需的服务。这将启动 `docker-compose.yml` 中的所有服务，包括： +啟動所需的服务。这將啟動 `docker-compose.yml` 中的所有服务，包括： - searxng - redis（由 redis 提供支持） - 前端 @@ -201,25 +201,25 @@ sudo ./start_services.sh # MacOS start ./start_services.cmd # Windows ``` -**选项 1:** 使用 CLI 界面运行。 +**選項 1:** 使用 CLI 界面運行。 ```sh python3 cli.py ``` -**选项 2:** 使用 Web 界面运行。 +**選項 2:** 使用 Web 界面運行。 注意：目前我們建議您使用 CLI 界面。Web 界面仍在積極開發中。 -启动后端服务。 +啟動後端服务。 ```sh python3 api.py ``` -访问 `http://localhost:3000/`，你应该会看到 Web 界面。 +访问 `http://localhost:3000/`，你应该會看到 Web 界面。 -请注意，目前 Web 界面不支持消息流式传输。 +请注意，目前 Web 界面不支持消息流式傳輸。 *如果你不知道如何開始，請參閱 **Usage** 部分* @@ -228,9 +228,9 @@ python3 api.py ## Usage （使用方法） -为确保 agenticSeek 在中文环境下正常工作，请确保在 config.ini 中设置语言选项。 +为确保 agenticSeek 在中文环境下正常工作，请确保在 config.ini 中設定语言選項。 languages = en zh -更多信息请参阅 Config 部分 +更多信息请参閱 Config 部分 確定所有的核心檔案都啟用了，也就是執行過這條命令 `./start_services.sh` 然後你就可以使用 `python3 cli.py` 來啟動 AgenticSeek 了！ @@ -287,11 +287,11 @@ python3 cli.py 所以我們希望你在使用時，能明確地表明你希望他要怎麼做，下面給你一個範例！ 你該說： -- 进行网络搜索，找出哪些国家最适合独自旅行 +- 進行網路搜索，找出哪些国家最适合獨自旅行 而不是說： -- 你知道哪些国家适合独自旅行？ +- 你知道哪些国家适合獨自旅行？ --- @@ -357,7 +357,7 @@ provider_server_address = x.x.x.x:3333 ## 語音轉文字 -请注意，目前语音转文字功能仅支持英语。 +请注意，目前語音轉文字功能僅支援英语。 預設狀況下，語音轉文字功能是停用的。若要啟用它，請在 `config.ini` 檔案中，將 `listen` 選項設為 `True`： @@ -538,13 +538,13 @@ https://googlechromelabs.github.io/chrome-for-testing/ **Q: 是否支持中文以外的语言？** -DeepSeek R1 天生会说中文 +DeepSeek R1 天生會说中文 但注意：代理路由系统只懂英文，所以必须通过 config.ini 的 languages 参数（如 languages = en zh）告诉系统： -如果不设置中文？后果可能是：你让它写代码，结果跳出来个"医生代理"（虽然我们根本没有这个代理... 但系统会一脸懵圈！） +如果不設定中文？後果可能是：你讓它寫代码，结果跳出来个"醫生代理"（虽然我们根本没有这个代理... 但系统會一脸懵圈！） -实际上会下载一个小型翻译模型来协助任务分配 +实际上會下载一个小型翻译模型来协助任务分配 ## 貢獻 @@ -556,8 +556,8 @@ DeepSeek R1 天生会说中文 ## 维护者: - > [Fosowl](https://github.com/Fosowl) | 巴黎时间 | （有时很忙） + > [Fosowl](https://github.com/Fosowl) | 巴黎時間 | （有时很忙） - > [https://github.com/antoineVIVIES](https://github.com/antoineVIVIES) | 台北时间 | （经常很忙） + > [https://github.com/antoineVIVIES](https://github.com/antoineVIVIES) | 台北時間 | （經常很忙） - > [steveh8758](https://github.com/steveh8758) | 台北时间 | （总是很忙） + > [steveh8758](https://github.com/steveh8758) | 台北時間 | （總是很忙） From ee6687df859e2385116f2e2da417a26def21c32e Mon Sep 17 00:00:00 2001 From: manra399 Date: Tue, 27 May 2025 13:55:52 +0100 Subject: [PATCH 03/14] Added Docker Ignore file. --- .dockerignore | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 .dockerignore diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..14aab39 --- /dev/null +++ b/.dockerignore @@ -0,0 +1,19 @@ +# Python cache files +__pycache__/ +*.py[cod] + +# Virtual environments +venv/ +.venv/ + +# Environment variables (secrets) +.env + +# Git metadata +.git/ + +# macOS Finder files +.DS_Store + +# Log files +*.log \ No newline at end of file From a53842b8b7301a15a98eaa76768a4a8173e80e69 Mon Sep 17 00:00:00 2001 From: martin legrand Date: Tue, 27 May 2025 19:29:03 +0200 Subject: [PATCH 04/14] update readme disclaimer --- README.md | 4 +- searxng/settings.yml.new | 2737 -------------------------------------- searxng/uwsgi.ini.new | 55 - 3 files changed, 3 insertions(+), 2793 deletions(-) delete mode 100644 searxng/settings.yml.new delete mode 100644 searxng/uwsgi.ini.new diff --git a/README.md b/README.md index f18ccd0..34612c5 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,9 @@ https://github.com/user-attachments/assets/b8ca60e9-7b3b-4533-840e-08f9ac426316 Disclaimer: This demo, including all the files that appear (e.g: CV_candidates.zip), are entirely fictional. We are not a corporation, we seek open-source contributors not candidates. -> 🛠️ **Work in Progress** – Looking for contributors! +> 🛠⚠️️ **Active Work in Progress** – Please note that Code/Bash is not dockerized yet but will be soon (see docker_deployement branch) - Do not deploy over network or production. + +> 🙏 Please also understand that this project began as a side experiment, with no roadmap and no expectations, we didn't expect to end in Github trending. Financial backing is exactly $1/month (shoutout to my single sponsor). Contributions, feedback, and patience are deeply appreciated. ## Installation diff --git a/searxng/settings.yml.new b/searxng/settings.yml.new deleted file mode 100644 index 131529b..0000000 --- a/searxng/settings.yml.new +++ /dev/null @@ -1,2737 +0,0 @@ -general: - # Debug mode, only for development. Is overwritten by ${SEARXNG_DEBUG} - debug: false - # displayed name - instance_name: "searxng" - # For example: https://example.com/privacy - privacypolicy_url: false - # use true to use your own donation page written in searx/info/en/donate.md - # use false to disable the donation link - donation_url: false - # mailto:contact@example.com - contact_url: false - # record stats - enable_metrics: true - # expose stats in open metrics format at /metrics - # leave empty to disable (no password set) - # open_metrics: - open_metrics: '' - -brand: - new_issue_url: https://github.com/searxng/searxng/issues/new - docs_url: https://docs.searxng.org/ - public_instances: https://searx.space - wiki_url: https://github.com/searxng/searxng/wiki - issue_url: https://github.com/searxng/searxng/issues - # custom: - # maintainer: "Jon Doe" - # # Custom entries in the footer: [title]: [link] - # links: - # Uptime: https://uptime.searxng.org/history/darmarit-org - # About: "https://searxng.org" - -search: - # Filter results. 0: None, 1: Moderate, 2: Strict - safe_search: 0 - # Existing autocomplete backends: "360search", "baidu", "brave", "dbpedia", "duckduckgo", "google", "yandex", - # "mwmbl", "seznam", "sogou", "stract", "swisscows", "quark", "qwant", "wikipedia" - - # leave blank to turn it off by default. - autocomplete: "" - # minimun characters to type before autocompleter starts - autocomplete_min: 4 - # backend for the favicon near URL in search results. - # Available resolvers: "allesedv", "duckduckgo", "google", "yandex" - leave blank to turn it off by default. - favicon_resolver: "" - # Default search language - leave blank to detect from browser information or - # use codes from 'languages.py' - default_lang: "auto" - # max_page: 0 # if engine supports paging, 0 means unlimited numbers of pages - # Available languages - # languages: - # - all - # - en - # - en-US - # - de - # - it-IT - # - fr - # - fr-BE - # ban time in seconds after engine errors - ban_time_on_fail: 5 - # max ban time in seconds after engine errors - max_ban_time_on_fail: 120 - suspended_times: - # Engine suspension time after error (in seconds; set to 0 to disable) - # For error "Access denied" and "HTTP error [402, 403]" - SearxEngineAccessDenied: 86400 - # For error "CAPTCHA" - SearxEngineCaptcha: 86400 - # For error "Too many request" and "HTTP error 429" - SearxEngineTooManyRequests: 3600 - # Cloudflare CAPTCHA - cf_SearxEngineCaptcha: 1296000 - cf_SearxEngineAccessDenied: 86400 - # ReCAPTCHA - recaptcha_SearxEngineCaptcha: 604800 - - # remove format to deny access, use lower case. - # formats: [html, csv, json, rss] - formats: - - html - -server: - # Is overwritten by ${SEARXNG_PORT} and ${SEARXNG_BIND_ADDRESS} - port: 8888 - bind_address: "127.0.0.1" - # public URL of the instance, to ensure correct inbound links. Is overwritten - # by ${SEARXNG_URL}. - base_url: / # "http://example.com/location" - # rate limit the number of request on the instance, block some bots. - # Is overwritten by ${SEARXNG_LIMITER} - limiter: false - # enable features designed only for public instances. - # Is overwritten by ${SEARXNG_PUBLIC_INSTANCE} - public_instance: false - - # If your instance owns a /etc/searxng/settings.yml file, then set the following - # values there. - - secret_key: "2b3888dcb9503a911f9ef053b7828803724a59815ce901774d9d73fa63603360" # Is overwritten by ${SEARXNG_SECRET} - # Proxy image results through SearXNG. Is overwritten by ${SEARXNG_IMAGE_PROXY} - image_proxy: false - # 1.0 and 1.1 are supported - http_protocol_version: "1.0" - # POST queries are more secure as they don't show up in history but may cause - # problems when using Firefox containers - method: "POST" - default_http_headers: - X-Content-Type-Options: nosniff - X-Download-Options: noopen - X-Robots-Tag: noindex, nofollow - Referrer-Policy: no-referrer - -redis: - # URL to connect redis database. Is overwritten by ${SEARXNG_REDIS_URL}. - # https://docs.searxng.org/admin/settings/settings_redis.html#settings-redis - url: false - -ui: - # Custom static path - leave it blank if you didn't change - static_path: "" - # Is overwritten by ${SEARXNG_STATIC_USE_HASH}. - static_use_hash: false - # Custom templates path - leave it blank if you didn't change - templates_path: "" - # query_in_title: When true, the result page's titles contains the query - # it decreases the privacy, since the browser can records the page titles. - query_in_title: false - # infinite_scroll: When true, automatically loads the next page when scrolling to bottom of the current page. - infinite_scroll: false - # ui theme - default_theme: simple - # center the results ? - center_alignment: false - # URL prefix of the internet archive, don't forget trailing slash (if needed). - # cache_url: "https://webcache.googleusercontent.com/search?q=cache:" - # Default interface locale - leave blank to detect from browser information or - # use codes from the 'locales' config section - default_locale: "" - # Open result links in a new tab by default - # results_on_new_tab: false - theme_args: - # style of simple theme: auto, light, dark - simple_style: auto - # Perform search immediately if a category selected. - # Disable to select multiple categories at once and start the search manually. - search_on_category_select: true - # Hotkeys: default or vim - hotkeys: default - # URL formatting: pretty, full or host - url_formatting: pretty - -# Lock arbitrary settings on the preferences page. -# -# preferences: -# lock: -# - categories -# - language -# - autocomplete -# - favicon -# - safesearch -# - method -# - doi_resolver -# - locale -# - theme -# - results_on_new_tab -# - infinite_scroll -# - search_on_category_select -# - method -# - image_proxy -# - query_in_title - -# searx supports result proxification using an external service: -# https://github.com/asciimoo/morty uncomment below section if you have running -# morty proxy the key is base64 encoded (keep the !!binary notation) -# Note: since commit af77ec3, morty accepts a base64 encoded key. -# -# result_proxy: -# url: http://127.0.0.1:3000/ -# # the key is a base64 encoded string, the YAML !!binary prefix is optional -# key: !!binary "your_morty_proxy_key" -# # [true|false] enable the "proxy" button next to each result -# proxify_results: true - -# communication with search engines -# -outgoing: - # default timeout in seconds, can be override by engine - request_timeout: 3.0 - # the maximum timeout in seconds - # max_request_timeout: 10.0 - # suffix of searx_useragent, could contain information like an email address - # to the administrator - useragent_suffix: "" - # The maximum number of concurrent connections that may be established. - pool_connections: 100 - # Allow the connection pool to maintain keep-alive connections below this - # point. - pool_maxsize: 20 - # See https://www.python-httpx.org/http2/ - enable_http2: true - # uncomment below section if you want to use a custom server certificate - # see https://www.python-httpx.org/advanced/#changing-the-verification-defaults - # and https://www.python-httpx.org/compatibility/#ssl-configuration - # verify: ~/.mitmproxy/mitmproxy-ca-cert.cer - # - # uncomment below section if you want to use a proxyq see: SOCKS proxies - # https://2.python-requests.org/en/latest/user/advanced/#proxies - # are also supported: see - # https://2.python-requests.org/en/latest/user/advanced/#socks - # - # proxies: - # all://: - # - http://proxy1:8080 - # - http://proxy2:8080 - # - # using_tor_proxy: true - # - # Extra seconds to add in order to account for the time taken by the proxy - # - # extra_proxy_timeout: 10 - # - # uncomment below section only if you have more than one network interface - # which can be the source of outgoing search requests - # - # source_ips: - # - 1.1.1.1 - # - 1.1.1.2 - # - fe80::/126 - -# Plugin configuration, for more details see -# https://docs.searxng.org/admin/settings/settings_plugins.html -# -plugins: - - searx.plugins.calculator.SXNGPlugin: - active: true - - searx.plugins.hash_plugin.SXNGPlugin: - active: true - - searx.plugins.self_info.SXNGPlugin: - active: true - - searx.plugins.unit_converter.SXNGPlugin: - active: true - - searx.plugins.ahmia_filter.SXNGPlugin: - active: true - - searx.plugins.hostnames.SXNGPlugin: - active: true - - searx.plugins.oa_doi_rewrite.SXNGPlugin: - active: false - - searx.plugins.tor_check.SXNGPlugin: - active: false - - searx.plugins.tracker_url_remover.SXNGPlugin: - active: false - - -# Configuration of the "Hostnames plugin": -# -# hostnames: -# replace: -# '(.*\.)?youtube\.com$': 'invidious.example.com' -# '(.*\.)?youtu\.be$': 'invidious.example.com' -# '(.*\.)?reddit\.com$': 'teddit.example.com' -# '(.*\.)?redd\.it$': 'teddit.example.com' -# '(www\.)?twitter\.com$': 'nitter.example.com' -# remove: -# - '(.*\.)?facebook.com$' -# low_priority: -# - '(.*\.)?google(\..*)?$' -# high_priority: -# - '(.*\.)?wikipedia.org$' -# -# Alternatively you can use external files for configuring the "Hostnames plugin": -# -# hostnames: -# replace: 'rewrite-hosts.yml' -# -# Content of 'rewrite-hosts.yml' (place the file in the same directory as 'settings.yml'): -# '(.*\.)?youtube\.com$': 'invidious.example.com' -# '(.*\.)?youtu\.be$': 'invidious.example.com' -# - -checker: - # disable checker when in debug mode - off_when_debug: true - - # use "scheduling: false" to disable scheduling - # scheduling: interval or int - - # to activate the scheduler: - # * uncomment "scheduling" section - # * add "cache2 = name=searxngcache,items=2000,blocks=2000,blocksize=4096,bitmap=1" - # to your uwsgi.ini - - # scheduling: - # start_after: [300, 1800] # delay to start the first run of the checker - # every: [86400, 90000] # how often the checker runs - - # additional tests: only for the YAML anchors (see the engines section) - # - additional_tests: - rosebud: &test_rosebud - matrix: - query: rosebud - lang: en - result_container: - - not_empty - - ['one_title_contains', 'citizen kane'] - test: - - unique_results - - android: &test_android - matrix: - query: ['android'] - lang: ['en', 'de', 'fr', 'zh-CN'] - result_container: - - not_empty - - ['one_title_contains', 'google'] - test: - - unique_results - - # tests: only for the YAML anchors (see the engines section) - tests: - infobox: &tests_infobox - infobox: - matrix: - query: ["linux", "new york", "bbc"] - result_container: - - has_infobox - -categories_as_tabs: - general: - images: - videos: - news: - map: - music: - it: - science: - files: - social media: - -engines: - - name: 360search - engine: 360search - shortcut: 360so - disabled: true - - - name: 360search videos - engine: 360search_videos - shortcut: 360sov - disabled: true - - - name: 9gag - engine: 9gag - shortcut: 9g - disabled: true - - - name: acfun - engine: acfun - shortcut: acf - disabled: true - - - name: adobe stock - engine: adobe_stock - shortcut: asi - categories: ["images"] - # https://docs.searxng.org/dev/engines/online/adobe_stock.html - adobe_order: relevance - adobe_content_types: ["photo", "illustration", "zip_vector", "template", "3d", "image"] - timeout: 6 - disabled: true - - - name: adobe stock video - engine: adobe_stock - shortcut: asv - network: adobe stock - categories: ["videos"] - adobe_order: relevance - adobe_content_types: ["video"] - timeout: 6 - disabled: true - - - name: adobe stock audio - engine: adobe_stock - shortcut: asa - network: adobe stock - categories: ["music"] - adobe_order: relevance - adobe_content_types: ["audio"] - timeout: 6 - disabled: true - - - name: alexandria - engine: json_engine - shortcut: alx - categories: general - paging: true - search_url: https://api.alexandria.org/?a=1&q={query}&p={pageno} - results_query: results - title_query: title - url_query: url - content_query: snippet - timeout: 1.5 - disabled: true - about: - website: https://alexandria.org/ - official_api_documentation: https://github.com/alexandria-org/alexandria-api/raw/master/README.md - use_official_api: true - require_api_key: false - results: JSON - - # - name: astrophysics data system - # engine: astrophysics_data_system - # sort: asc - # weight: 5 - # categories: [science] - # api_key: your-new-key - # shortcut: ads - - - name: alpine linux packages - engine: alpinelinux - disabled: true - shortcut: alp - - - name: annas archive - engine: annas_archive - disabled: true - shortcut: aa - - # - name: annas articles - # engine: annas_archive - # shortcut: aaa - # # https://docs.searxng.org/dev/engines/online/annas_archive.html - # aa_content: 'magazine' # book_fiction, book_unknown, book_nonfiction, book_comic - # aa_ext: 'pdf' # pdf, epub, .. - # aa_sort: oldest' # newest, oldest, largest, smallest - - - name: apk mirror - engine: apkmirror - timeout: 4.0 - shortcut: apkm - disabled: true - - - name: apple app store - engine: apple_app_store - shortcut: aps - disabled: true - - # Requires Tor - - name: ahmia - engine: ahmia - categories: onions - enable_http: true - shortcut: ah - - - name: anaconda - engine: xpath - paging: true - first_page_num: 0 - search_url: https://anaconda.org/search?q={query}&page={pageno} - results_xpath: //tbody/tr - url_xpath: ./td/h5/a[last()]/@href - title_xpath: ./td/h5 - content_xpath: ./td[h5]/text() - categories: it - timeout: 6.0 - shortcut: conda - disabled: true - - - name: arch linux wiki - engine: archlinux - shortcut: al - - - name: nixos wiki - engine: mediawiki - shortcut: nixw - base_url: https://wiki.nixos.org/ - search_type: text - disabled: true - categories: [it, software wikis] - - - name: artic - engine: artic - shortcut: arc - timeout: 4.0 - - - name: arxiv - engine: arxiv - shortcut: arx - timeout: 4.0 - - - name: ask - engine: ask - shortcut: ask - disabled: true - - # tmp suspended: dh key too small - # - name: base - # engine: base - # shortcut: bs - - - name: bandcamp - engine: bandcamp - shortcut: bc - categories: music - - - name: baidu - baidu_category: general - categories: [general] - engine: baidu - shortcut: bd - disabled: true - - - name: baidu images - baidu_category: images - categories: [images] - engine: baidu - shortcut: bdi - disabled: true - - - name: baidu kaifa - baidu_category: it - categories: [it] - engine: baidu - shortcut: bdk - disabled: true - - - name: wikipedia - engine: wikipedia - shortcut: wp - # add "list" to the array to get results in the results list - display_type: ["infobox"] - categories: [general] - - - name: bilibili - engine: bilibili - shortcut: bil - disabled: true - - - name: bing - engine: bing - shortcut: bi - disabled: true - - - name: bing images - engine: bing_images - shortcut: bii - - - name: bing news - engine: bing_news - shortcut: bin - - - name: bing videos - engine: bing_videos - shortcut: biv - - - name: bitchute - engine: bitchute - shortcut: bit - disabled: true - - - name: bitbucket - engine: xpath - paging: true - search_url: https://bitbucket.org/repo/all/{pageno}?name={query} - url_xpath: //article[@class="repo-summary"]//a[@class="repo-link"]/@href - title_xpath: //article[@class="repo-summary"]//a[@class="repo-link"] - content_xpath: //article[@class="repo-summary"]/p - categories: [it, repos] - timeout: 4.0 - disabled: true - shortcut: bb - about: - website: https://bitbucket.org/ - wikidata_id: Q2493781 - official_api_documentation: https://developer.atlassian.com/bitbucket - use_official_api: false - require_api_key: false - results: HTML - - - name: bpb - engine: bpb - shortcut: bpb - disabled: true - - - name: btdigg - engine: btdigg - shortcut: bt - disabled: true - - - name: openverse - engine: openverse - categories: images - shortcut: opv - - - name: media.ccc.de - engine: ccc_media - shortcut: c3tv - # We don't set language: de here because media.ccc.de is not just - # for a German audience. It contains many English videos and many - # German videos have English subtitles. - disabled: true - - - name: chefkoch - engine: chefkoch - shortcut: chef - # to show premium or plus results too: - # skip_premium: false - - - name: chinaso news - chinaso_category: news - engine: chinaso - shortcut: chinaso - disabled: true - - - name: chinaso images - chinaso_category: images - engine: chinaso - shortcut: chinasoi - disabled: true - - - name: chinaso videos - chinaso_category: videos - engine: chinaso - shortcut: chinasov - disabled: true - - - name: cloudflareai - engine: cloudflareai - shortcut: cfai - # get api token and accont id from https://developers.cloudflare.com/workers-ai/get-started/rest-api/ - cf_account_id: 'your_cf_accout_id' - cf_ai_api: 'your_cf_api' - # create your ai gateway by https://developers.cloudflare.com/ai-gateway/get-started/creating-gateway/ - cf_ai_gateway: 'your_cf_ai_gateway_name' - # find the model name from https://developers.cloudflare.com/workers-ai/models/#text-generation - cf_ai_model: 'ai_model_name' - # custom your preferences - # cf_ai_model_display_name: 'Cloudflare AI' - # cf_ai_model_assistant: 'prompts_for_assistant_role' - # cf_ai_model_system: 'prompts_for_system_role' - timeout: 30 - disabled: true - - # - name: core.ac.uk - # engine: core - # categories: science - # shortcut: cor - # # get your API key from: https://core.ac.uk/api-keys/register/ - # api_key: 'unset' - - - name: cppreference - engine: cppreference - shortcut: cpp - paging: false - disabled: true - - - name: crossref - engine: crossref - shortcut: cr - timeout: 30 - disabled: true - - - name: crowdview - engine: json_engine - shortcut: cv - categories: general - paging: false - search_url: https://crowdview-next-js.onrender.com/api/search-v3?query={query} - results_query: results - url_query: link - title_query: title - content_query: snippet - title_html_to_text: true - content_html_to_text: true - disabled: true - about: - website: https://crowdview.ai/ - - - name: yep - engine: yep - shortcut: yep - categories: general - search_type: web - timeout: 5 - disabled: true - - - name: yep images - engine: yep - shortcut: yepi - categories: images - search_type: images - disabled: true - - - name: yep news - engine: yep - shortcut: yepn - categories: news - search_type: news - disabled: true - - - name: curlie - engine: xpath - shortcut: cl - categories: general - disabled: true - paging: true - lang_all: '' - search_url: https://curlie.org/search?q={query}&lang={lang}&start={pageno}&stime=92452189 - page_size: 20 - results_xpath: //div[@id="site-list-content"]/div[@class="site-item"] - url_xpath: ./div[@class="title-and-desc"]/a/@href - title_xpath: ./div[@class="title-and-desc"]/a/div - content_xpath: ./div[@class="title-and-desc"]/div[@class="site-descr"] - about: - website: https://curlie.org/ - wikidata_id: Q60715723 - use_official_api: false - require_api_key: false - results: HTML - - - name: currency - engine: currency_convert - categories: general - shortcut: cc - - - name: deezer - engine: deezer - shortcut: dz - disabled: true - - - name: destatis - engine: destatis - shortcut: destat - disabled: true - - - name: deviantart - engine: deviantart - shortcut: da - timeout: 3.0 - - - name: ddg definitions - engine: duckduckgo_definitions - shortcut: ddd - weight: 2 - disabled: true - tests: *tests_infobox - - # cloudflare protected - # - name: digbt - # engine: digbt - # shortcut: dbt - # timeout: 6.0 - # disabled: true - - - name: docker hub - engine: docker_hub - shortcut: dh - categories: [it, packages] - - - name: encyclosearch - engine: json_engine - shortcut: es - categories: general - paging: true - search_url: https://encyclosearch.org/encyclosphere/search?q={query}&page={pageno}&resultsPerPage=15 - results_query: Results - url_query: SourceURL - title_query: Title - content_query: Description - disabled: true - about: - website: https://encyclosearch.org - official_api_documentation: https://encyclosearch.org/docs/#/rest-api - use_official_api: true - require_api_key: false - results: JSON - - - name: erowid - engine: xpath - paging: true - first_page_num: 0 - page_size: 30 - search_url: https://www.erowid.org/search.php?q={query}&s={pageno} - url_xpath: //dl[@class="results-list"]/dt[@class="result-title"]/a/@href - title_xpath: //dl[@class="results-list"]/dt[@class="result-title"]/a/text() - content_xpath: //dl[@class="results-list"]/dd[@class="result-details"] - categories: [] - shortcut: ew - disabled: true - about: - website: https://www.erowid.org/ - wikidata_id: Q1430691 - official_api_documentation: - use_official_api: false - require_api_key: false - results: HTML - - # - name: elasticsearch - # shortcut: els - # engine: elasticsearch - # base_url: http://localhost:9200 - # username: elastic - # password: changeme - # index: my-index - # enable_http: true - # # available options: match, simple_query_string, term, terms, custom - # query_type: match - # # if query_type is set to custom, provide your query here - # # custom_query_json: {"query":{"match_all": {}}} - # # show_metadata: false - # disabled: true - - - name: wikidata - engine: wikidata - shortcut: wd - timeout: 3.0 - weight: 2 - # add "list" to the array to get results in the results list - display_type: ["infobox"] - tests: *tests_infobox - categories: [general] - - - name: duckduckgo - engine: duckduckgo - shortcut: ddg - - - name: duckduckgo images - engine: duckduckgo_extra - categories: [images, web] - ddg_category: images - shortcut: ddi - disabled: true - - - name: duckduckgo videos - engine: duckduckgo_extra - categories: [videos, web] - ddg_category: videos - shortcut: ddv - disabled: true - - - name: duckduckgo news - engine: duckduckgo_extra - categories: [news, web] - ddg_category: news - shortcut: ddn - disabled: true - - - name: duckduckgo weather - engine: duckduckgo_weather - shortcut: ddw - disabled: true - - - name: apple maps - engine: apple_maps - shortcut: apm - disabled: true - timeout: 5.0 - - - name: emojipedia - engine: emojipedia - timeout: 4.0 - shortcut: em - disabled: true - - - name: tineye - engine: tineye - shortcut: tin - timeout: 9.0 - disabled: true - - - name: etymonline - engine: xpath - paging: true - search_url: https://etymonline.com/search?page={pageno}&q={query} - url_xpath: //a[contains(@class, "word__name--")]/@href - title_xpath: //a[contains(@class, "word__name--")] - content_xpath: //section[contains(@class, "word__defination")] - first_page_num: 1 - shortcut: et - categories: [dictionaries] - about: - website: https://www.etymonline.com/ - wikidata_id: Q1188617 - official_api_documentation: - use_official_api: false - require_api_key: false - results: HTML - - # - name: ebay - # engine: ebay - # shortcut: eb - # base_url: 'https://www.ebay.com' - # disabled: true - # timeout: 5 - - - name: 1x - engine: www1x - shortcut: 1x - timeout: 3.0 - disabled: true - - - name: fdroid - engine: fdroid - shortcut: fd - disabled: true - - - name: findthatmeme - engine: findthatmeme - shortcut: ftm - disabled: true - - - name: flickr - categories: images - shortcut: fl - # You can use the engine using the official stable API, but you need an API - # key, see: https://www.flickr.com/services/apps/create/ - # engine: flickr - # api_key: 'apikey' # required! - # Or you can use the html non-stable engine, activated by default - engine: flickr_noapi - - - name: free software directory - engine: mediawiki - shortcut: fsd - categories: [it, software wikis] - base_url: https://directory.fsf.org/ - search_type: title - timeout: 5.0 - disabled: true - about: - website: https://directory.fsf.org/ - wikidata_id: Q2470288 - - # - name: freesound - # engine: freesound - # shortcut: fnd - # disabled: true - # timeout: 15.0 - # API key required, see: https://freesound.org/docs/api/overview.html - # api_key: MyAPIkey - - - name: frinkiac - engine: frinkiac - shortcut: frk - disabled: true - - - name: fyyd - engine: fyyd - shortcut: fy - timeout: 8.0 - disabled: true - - - name: geizhals - engine: geizhals - shortcut: geiz - disabled: true - - - name: genius - engine: genius - shortcut: gen - - - name: gentoo - engine: mediawiki - shortcut: ge - categories: ["it", "software wikis"] - base_url: "https://wiki.gentoo.org/" - api_path: "api.php" - search_type: text - timeout: 10 - - - name: gitlab - engine: gitlab - base_url: https://gitlab.com - shortcut: gl - disabled: true - about: - website: https://gitlab.com/ - wikidata_id: Q16639197 - - # - name: gnome - # engine: gitlab - # base_url: https://gitlab.gnome.org - # shortcut: gn - # about: - # website: https://gitlab.gnome.org - # wikidata_id: Q44316 - - - name: github - engine: github - shortcut: gh - - - name: codeberg - # https://docs.searxng.org/dev/engines/online/gitea.html - engine: gitea - base_url: https://codeberg.org - shortcut: cb - disabled: true - - - name: gitea.com - engine: gitea - base_url: https://gitea.com - shortcut: gitea - disabled: true - - - name: goodreads - engine: goodreads - shortcut: good - timeout: 4.0 - disabled: true - - - name: google - engine: google - shortcut: go - # additional_tests: - # android: *test_android - - - name: google images - engine: google_images - shortcut: goi - # additional_tests: - # android: *test_android - # dali: - # matrix: - # query: ['Dali Christ'] - # lang: ['en', 'de', 'fr', 'zh-CN'] - # result_container: - # - ['one_title_contains', 'Salvador'] - - - name: google news - engine: google_news - shortcut: gon - # additional_tests: - # android: *test_android - - - name: google videos - engine: google_videos - shortcut: gov - # additional_tests: - # android: *test_android - - - name: google scholar - engine: google_scholar - shortcut: gos - - - name: google play apps - engine: google_play - categories: [files, apps] - shortcut: gpa - play_categ: apps - disabled: true - - - name: google play movies - engine: google_play - categories: videos - shortcut: gpm - play_categ: movies - disabled: true - - - name: material icons - engine: material_icons - categories: images - shortcut: mi - disabled: true - - - name: habrahabr - engine: xpath - paging: true - search_url: https://habr.com/en/search/page{pageno}/?q={query} - results_xpath: //article[contains(@class, "tm-articles-list__item")] - url_xpath: .//a[@class="tm-title__link"]/@href - title_xpath: .//a[@class="tm-title__link"] - content_xpath: .//div[contains(@class, "article-formatted-body")] - categories: it - timeout: 4.0 - disabled: true - shortcut: habr - about: - website: https://habr.com/ - wikidata_id: Q4494434 - official_api_documentation: https://habr.com/en/docs/help/api/ - use_official_api: false - require_api_key: false - results: HTML - - - name: hackernews - engine: hackernews - shortcut: hn - disabled: true - - - name: hex - engine: hex - shortcut: hex - disabled: true - # Valid values: name inserted_at updated_at total_downloads recent_downloads - sort_criteria: "recent_downloads" - page_size: 10 - - - name: crates.io - engine: crates - shortcut: crates - disabled: true - timeout: 6.0 - - - name: hoogle - engine: xpath - search_url: https://hoogle.haskell.org/?hoogle={query} - results_xpath: '//div[@class="result"]' - title_xpath: './/div[@class="ans"]//a' - url_xpath: './/div[@class="ans"]//a/@href' - content_xpath: './/div[@class="from"]' - page_size: 20 - categories: [it, packages] - shortcut: ho - about: - website: https://hoogle.haskell.org/ - wikidata_id: Q34010 - official_api_documentation: https://hackage.haskell.org/api - use_official_api: false - require_api_key: false - results: JSON - - - name: il post - engine: il_post - shortcut: pst - disabled: true - - - name: imdb - engine: imdb - shortcut: imdb - timeout: 6.0 - disabled: true - - - name: imgur - engine: imgur - shortcut: img - disabled: true - - - name: ina - engine: ina - shortcut: in - timeout: 6.0 - disabled: true - - - name: invidious - engine: invidious - # Instanes will be selected randomly, see https://api.invidious.io/ for - # instances that are stable (good uptime) and close to you. - base_url: - - https://invidious.adminforge.de - - https://inv.nadeko.net - shortcut: iv - timeout: 3.0 - disabled: true - - - name: ipernity - engine: ipernity - shortcut: ip - disabled: true - - - name: iqiyi - engine: iqiyi - shortcut: iq - disabled: true - - - name: jisho - engine: jisho - shortcut: js - timeout: 3.0 - disabled: true - - - name: kickass - engine: kickass - base_url: - - https://kickasstorrents.to - - https://kickasstorrents.cr - - https://kickasstorrent.cr - - https://kickass.sx - - https://kat.am - shortcut: kc - timeout: 4.0 - - - name: lemmy communities - engine: lemmy - lemmy_type: Communities - shortcut: leco - - - name: lemmy users - engine: lemmy - network: lemmy communities - lemmy_type: Users - shortcut: leus - - - name: lemmy posts - engine: lemmy - network: lemmy communities - lemmy_type: Posts - shortcut: lepo - - - name: lemmy comments - engine: lemmy - network: lemmy communities - lemmy_type: Comments - shortcut: lecom - - - name: library genesis - engine: xpath - # search_url: https://libgen.is/search.php?req={query} - search_url: https://libgen.rs/search.php?req={query} - url_xpath: //a[contains(@href,"book/index.php?md5")]/@href - title_xpath: //a[contains(@href,"book/")]/text()[1] - content_xpath: //td/a[1][contains(@href,"=author")]/text() - categories: files - timeout: 7.0 - disabled: true - shortcut: lg - about: - website: https://libgen.fun/ - wikidata_id: Q22017206 - official_api_documentation: - use_official_api: false - require_api_key: false - results: HTML - - - name: z-library - engine: zlibrary - shortcut: zlib - categories: files - timeout: 7.0 - - - name: library of congress - engine: loc - shortcut: loc - categories: images - - - name: libretranslate - engine: libretranslate - # https://github.com/LibreTranslate/LibreTranslate?tab=readme-ov-file#mirrors - base_url: - - https://libretranslate.com/translate - # api_key: abc123 - shortcut: lt - disabled: true - - - name: lingva - engine: lingva - shortcut: lv - # set lingva instance in url, by default it will use the official instance - # url: https://lingva.thedaviddelta.com - - - name: lobste.rs - engine: xpath - search_url: https://lobste.rs/search?q={query}&what=stories&order=relevance - results_xpath: //li[contains(@class, "story")] - url_xpath: .//a[@class="u-url"]/@href - title_xpath: .//a[@class="u-url"] - content_xpath: .//a[@class="domain"] - categories: it - shortcut: lo - timeout: 5.0 - disabled: true - about: - website: https://lobste.rs/ - wikidata_id: Q60762874 - official_api_documentation: - use_official_api: false - require_api_key: false - results: HTML - - - name: mastodon users - engine: mastodon - mastodon_type: accounts - base_url: https://mastodon.social - shortcut: mau - - - name: mastodon hashtags - engine: mastodon - mastodon_type: hashtags - base_url: https://mastodon.social - shortcut: mah - - # - name: matrixrooms - # engine: mrs - # # https://docs.searxng.org/dev/engines/online/mrs.html - # # base_url: https://mrs-api-host - # shortcut: mtrx - # disabled: true - - - name: mdn - shortcut: mdn - engine: json_engine - categories: [it] - paging: true - search_url: https://developer.mozilla.org/api/v1/search?q={query}&page={pageno} - results_query: documents - url_query: mdn_url - url_prefix: https://developer.mozilla.org - title_query: title - content_query: summary - about: - website: https://developer.mozilla.org - wikidata_id: Q3273508 - official_api_documentation: null - use_official_api: false - require_api_key: false - results: JSON - - - name: metacpan - engine: metacpan - shortcut: cpan - disabled: true - number_of_results: 20 - - # https://docs.searxng.org/dev/engines/offline/search-indexer-engines.html#module-searx.engines.meilisearch - # - name: meilisearch - # engine: meilisearch - # shortcut: mes - # enable_http: true - # base_url: http://localhost:7700 - # index: my-index - # auth_key: Bearer XXXX - - - name: microsoft learn - engine: microsoft_learn - shortcut: msl - disabled: true - - - name: mixcloud - engine: mixcloud - shortcut: mc - - # MongoDB engine - # Required dependency: pymongo - # - name: mymongo - # engine: mongodb - # shortcut: md - # exact_match_only: false - # host: '127.0.0.1' - # port: 27017 - # enable_http: true - # results_per_page: 20 - # database: 'business' - # collection: 'reviews' # name of the db collection - # key: 'name' # key in the collection to search for - - - name: mozhi - engine: mozhi - base_url: - - https://mozhi.aryak.me - - https://translate.bus-hit.me - - https://nyc1.mz.ggtyler.dev - # mozhi_engine: google - see https://mozhi.aryak.me for supported engines - timeout: 4.0 - shortcut: mz - disabled: true - - - name: mwmbl - engine: mwmbl - # api_url: https://api.mwmbl.org - shortcut: mwm - disabled: true - - - name: niconico - engine: niconico - shortcut: nico - disabled: true - - - name: npm - engine: npm - shortcut: npm - timeout: 5.0 - disabled: true - - - name: nyaa - engine: nyaa - shortcut: nt - disabled: true - - - name: mankier - engine: json_engine - search_url: https://www.mankier.com/api/v2/mans/?q={query} - results_query: results - url_query: url - title_query: name - content_query: description - categories: it - shortcut: man - about: - website: https://www.mankier.com/ - official_api_documentation: https://www.mankier.com/api - use_official_api: true - require_api_key: false - results: JSON - - # read https://docs.searxng.org/dev/engines/online/mullvad_leta.html - # - name: mullvadleta - # engine: mullvad_leta - # leta_engine: google # choose one of the following: google, brave - # use_cache: true # Only 100 non-cache searches per day, suggested only for private instances - # search_url: https://leta.mullvad.net - # categories: [general, web] - # shortcut: ml - - - name: odysee - engine: odysee - shortcut: od - disabled: true - - - name: ollama - engine: ollama - shortcut: ollama - disabled: true - - - name: openairedatasets - engine: json_engine - paging: true - search_url: https://api.openaire.eu/search/datasets?format=json&page={pageno}&size=10&title={query} - results_query: response/results/result - url_query: metadata/oaf:entity/oaf:result/children/instance/webresource/url/$ - title_query: metadata/oaf:entity/oaf:result/title/$ - content_query: metadata/oaf:entity/oaf:result/description/$ - content_html_to_text: true - categories: "science" - shortcut: oad - timeout: 5.0 - about: - website: https://www.openaire.eu/ - wikidata_id: Q25106053 - official_api_documentation: https://api.openaire.eu/ - use_official_api: false - require_api_key: false - results: JSON - - - name: openairepublications - engine: json_engine - paging: true - search_url: https://api.openaire.eu/search/publications?format=json&page={pageno}&size=10&title={query} - results_query: response/results/result - url_query: metadata/oaf:entity/oaf:result/children/instance/webresource/url/$ - title_query: metadata/oaf:entity/oaf:result/title/$ - content_query: metadata/oaf:entity/oaf:result/description/$ - content_html_to_text: true - categories: science - shortcut: oap - timeout: 5.0 - about: - website: https://www.openaire.eu/ - wikidata_id: Q25106053 - official_api_documentation: https://api.openaire.eu/ - use_official_api: false - require_api_key: false - results: JSON - - - name: openclipart - engine: openclipart - shortcut: ocl - inactive: true - disabled: true - timeout: 30 - - - name: openlibrary - engine: openlibrary - shortcut: ol - timeout: 5 - disabled: true - - - name: openmeteo - engine: open_meteo - shortcut: om - disabled: true - - # - name: opensemanticsearch - # engine: opensemantic - # shortcut: oss - # base_url: 'http://localhost:8983/solr/opensemanticsearch/' - - - name: openstreetmap - engine: openstreetmap - shortcut: osm - - - name: openrepos - engine: xpath - paging: true - search_url: https://openrepos.net/search/node/{query}?page={pageno} - url_xpath: //li[@class="search-result"]//h3[@class="title"]/a/@href - title_xpath: //li[@class="search-result"]//h3[@class="title"]/a - content_xpath: //li[@class="search-result"]//div[@class="search-snippet-info"]//p[@class="search-snippet"] - categories: files - timeout: 4.0 - disabled: true - shortcut: or - about: - website: https://openrepos.net/ - wikidata_id: - official_api_documentation: - use_official_api: false - require_api_key: false - results: HTML - - - name: packagist - engine: json_engine - paging: true - search_url: https://packagist.org/search.json?q={query}&page={pageno} - results_query: results - url_query: url - title_query: name - content_query: description - categories: [it, packages] - disabled: true - timeout: 5.0 - shortcut: pack - about: - website: https://packagist.org - wikidata_id: Q108311377 - official_api_documentation: https://packagist.org/apidoc - use_official_api: true - require_api_key: false - results: JSON - - - name: pdbe - engine: pdbe - shortcut: pdb - # Hide obsolete PDB entries. Default is not to hide obsolete structures - # hide_obsolete: false - - - name: photon - engine: photon - shortcut: ph - - - name: pinterest - engine: pinterest - shortcut: pin - - - name: piped - engine: piped - shortcut: ppd - categories: videos - piped_filter: videos - timeout: 3.0 - - # URL to use as link and for embeds - frontend_url: https://srv.piped.video - # Instance will be selected randomly, for more see https://piped-instances.kavin.rocks/ - backend_url: - - https://pipedapi.adminforge.de - - https://pipedapi.nosebs.ru - - https://pipedapi.ducks.party - - https://pipedapi.reallyaweso.me - - https://api.piped.private.coffee - - https://pipedapi.darkness.services - - - name: piped.music - engine: piped - network: piped - shortcut: ppdm - categories: music - piped_filter: music_songs - timeout: 3.0 - - - name: piratebay - engine: piratebay - shortcut: tpb - # You may need to change this URL to a proxy if piratebay is blocked in your - # country - url: https://thepiratebay.org/ - timeout: 3.0 - - - name: pixiv - shortcut: pv - engine: pixiv - disabled: true - inactive: true - pixiv_image_proxies: - - https://pximg.example.org - # A proxy is required to load the images. Hosting an image proxy server - # for Pixiv: - # --> https://pixivfe.pages.dev/hosting-image-proxy-server/ - # Proxies from public instances. Ask the public instances owners if they - # agree to receive traffic from SearXNG! - # --> https://codeberg.org/VnPower/PixivFE#instances - # --> https://github.com/searxng/searxng/pull/3192#issuecomment-1941095047 - # image proxy of https://pixiv.cat - # - https://i.pixiv.cat - # image proxy of https://www.pixiv.pics - # - https://pximg.cocomi.eu.org - # image proxy of https://pixivfe.exozy.me - # - https://pximg.exozy.me - # image proxy of https://pixivfe.ducks.party - # - https://pixiv.ducks.party - # image proxy of https://pixiv.perennialte.ch - # - https://pximg.perennialte.ch - - - name: podcastindex - engine: podcastindex - shortcut: podcast - - # Required dependency: psychopg2 - # - name: postgresql - # engine: postgresql - # database: postgres - # username: postgres - # password: postgres - # limit: 10 - # query_str: 'SELECT * from my_table WHERE my_column = %(query)s' - # shortcut : psql - - - name: presearch - engine: presearch - search_type: search - categories: [general, web] - shortcut: ps - timeout: 4.0 - disabled: true - - - name: presearch images - engine: presearch - network: presearch - search_type: images - categories: [images, web] - timeout: 4.0 - shortcut: psimg - disabled: true - - - name: presearch videos - engine: presearch - network: presearch - search_type: videos - categories: [general, web] - timeout: 4.0 - shortcut: psvid - disabled: true - - - name: presearch news - engine: presearch - network: presearch - search_type: news - categories: [news, web] - timeout: 4.0 - shortcut: psnews - disabled: true - - - name: pub.dev - engine: xpath - shortcut: pd - search_url: https://pub.dev/packages?q={query}&page={pageno} - paging: true - results_xpath: //div[contains(@class,"packages-item")] - url_xpath: ./div/h3/a/@href - title_xpath: ./div/h3/a - content_xpath: ./div/div/div[contains(@class,"packages-description")]/span - categories: [packages, it] - timeout: 3.0 - disabled: true - first_page_num: 1 - about: - website: https://pub.dev/ - official_api_documentation: https://pub.dev/help/api - use_official_api: false - require_api_key: false - results: HTML - - - name: public domain image archive - engine: public_domain_image_archive - shortcut: pdia - - - name: pubmed - engine: pubmed - shortcut: pub - timeout: 3.0 - - - name: pypi - shortcut: pypi - engine: pypi - - - name: quark - quark_category: general - categories: [general] - engine: quark - shortcut: qk - disabled: true - - - name: quark images - quark_category: images - categories: [images] - engine: quark - shortcut: qki - disabled: true - - - name: qwant - qwant_categ: web - engine: qwant - shortcut: qw - categories: [general, web] - additional_tests: - rosebud: *test_rosebud - - - name: qwant news - qwant_categ: news - engine: qwant - shortcut: qwn - categories: news - network: qwant - - - name: qwant images - qwant_categ: images - engine: qwant - shortcut: qwi - categories: [images, web] - network: qwant - - - name: qwant videos - qwant_categ: videos - engine: qwant - shortcut: qwv - categories: [videos, web] - network: qwant - - # - name: library - # engine: recoll - # shortcut: lib - # base_url: 'https://recoll.example.org/' - # search_dir: '' - # mount_prefix: /export - # dl_prefix: 'https://download.example.org' - # timeout: 30.0 - # categories: files - # disabled: true - - # - name: recoll library reference - # engine: recoll - # base_url: 'https://recoll.example.org/' - # search_dir: reference - # mount_prefix: /export - # dl_prefix: 'https://download.example.org' - # shortcut: libr - # timeout: 30.0 - # categories: files - # disabled: true - - - name: radio browser - engine: radio_browser - shortcut: rb - - - name: reddit - engine: reddit - shortcut: re - page_size: 25 - disabled: true - - - name: reuters - engine: reuters - shortcut: reu - # https://docs.searxng.org/dev/engines/online/reuters.html - # sort_order = "relevance" - - - name: right dao - engine: xpath - paging: true - page_size: 12 - search_url: https://rightdao.com/search?q={query}&start={pageno} - results_xpath: //div[contains(@class, "description")] - url_xpath: ../div[contains(@class, "title")]/a/@href - title_xpath: ../div[contains(@class, "title")] - content_xpath: . - categories: general - shortcut: rd - disabled: true - about: - website: https://rightdao.com/ - use_official_api: false - require_api_key: false - results: HTML - - - name: rottentomatoes - engine: rottentomatoes - shortcut: rt - disabled: true - - # Required dependency: redis - # - name: myredis - # shortcut : rds - # engine: redis_server - # exact_match_only: false - # host: '127.0.0.1' - # port: 6379 - # enable_http: true - # password: '' - # db: 0 - - # tmp suspended: bad certificate - # - name: scanr structures - # shortcut: scs - # engine: scanr_structures - # disabled: true - - - name: searchmysite - engine: xpath - shortcut: sms - categories: general - paging: true - search_url: https://searchmysite.net/search/?q={query}&page={pageno} - results_xpath: //div[contains(@class,'search-result')] - url_xpath: .//a[contains(@class,'result-link')]/@href - title_xpath: .//span[contains(@class,'result-title-txt')]/text() - content_xpath: ./p[@id='result-hightlight'] - disabled: true - about: - website: https://searchmysite.net - - - name: selfhst icons - engine: selfhst - shortcut: si - inactive: true - disabled: true - - - name: sepiasearch - engine: sepiasearch - shortcut: sep - - - name: sogou - engine: sogou - shortcut: sogou - disabled: true - - - name: sogou images - engine: sogou_images - shortcut: sogoui - disabled: true - - - name: sogou videos - engine: sogou_videos - shortcut: sogouv - disabled: true - - - name: sogou wechat - engine: sogou_wechat - shortcut: sogouw - disabled: true - - - name: soundcloud - engine: soundcloud - shortcut: sc - - - name: stackoverflow - engine: stackexchange - shortcut: st - api_site: 'stackoverflow' - categories: [it, q&a] - - - name: askubuntu - engine: stackexchange - shortcut: ubuntu - api_site: 'askubuntu' - categories: [it, q&a] - - - name: superuser - engine: stackexchange - shortcut: su - api_site: 'superuser' - categories: [it, q&a] - - - name: discuss.python - engine: discourse - shortcut: dpy - base_url: 'https://discuss.python.org' - categories: [it, q&a] - disabled: true - - - name: caddy.community - engine: discourse - shortcut: caddy - base_url: 'https://caddy.community' - categories: [it, q&a] - disabled: true - - - name: pi-hole.community - engine: discourse - shortcut: pi - categories: [it, q&a] - base_url: 'https://discourse.pi-hole.net' - disabled: true - - - name: searchcode code - engine: searchcode_code - shortcut: scc - disabled: true - - # - name: searx - # engine: searx_engine - # shortcut: se - # instance_urls : - # - http://127.0.0.1:8888/ - # - ... - # disabled: true - - - name: semantic scholar - engine: semantic_scholar - disabled: true - shortcut: se - - # Spotify needs API credentials - # - name: spotify - # engine: spotify - # shortcut: stf - # api_client_id: ******* - # api_client_secret: ******* - - # - name: solr - # engine: solr - # shortcut: slr - # base_url: http://localhost:8983 - # collection: collection_name - # sort: '' # sorting: asc or desc - # field_list: '' # comma separated list of field names to display on the UI - # default_fields: '' # default field to query - # query_fields: '' # query fields - # enable_http: true - - # - name: springer nature - # engine: springer - # # get your API key from: https://dev.springernature.com/signup - # # working API key, for test & debug: "a69685087d07eca9f13db62f65b8f601" - # api_key: 'unset' - # shortcut: springer - # timeout: 15.0 - - - name: startpage - engine: startpage - shortcut: sp - startpage_categ: web - categories: [general, web] - additional_tests: - rosebud: *test_rosebud - - - name: startpage news - engine: startpage - startpage_categ: news - categories: [news, web] - shortcut: spn - - - name: startpage images - engine: startpage - startpage_categ: images - categories: [images, web] - shortcut: spi - - - name: tokyotoshokan - engine: tokyotoshokan - shortcut: tt - timeout: 6.0 - disabled: true - - - name: solidtorrents - engine: solidtorrents - shortcut: solid - timeout: 4.0 - base_url: - - https://solidtorrents.to - - https://bitsearch.to - - # For this demo of the sqlite engine download: - # https://liste.mediathekview.de/filmliste-v2.db.bz2 - # and unpack into searx/data/filmliste-v2.db - # Query to test: "!mediathekview concert" - # - # - name: mediathekview - # engine: sqlite - # shortcut: mediathekview - # categories: [general, videos] - # result_type: MainResult - # database: searx/data/filmliste-v2.db - # query_str: >- - # SELECT title || ' (' || time(duration, 'unixepoch') || ')' AS title, - # COALESCE( NULLIF(url_video_hd,''), NULLIF(url_video_sd,''), url_video) AS url, - # description AS content - # FROM film - # WHERE title LIKE :wildcard OR description LIKE :wildcard - # ORDER BY duration DESC - - - name: tagesschau - engine: tagesschau - # when set to false, display URLs from Tagesschau, and not the actual source - # (e.g. NDR, WDR, SWR, HR, ...) - use_source_url: true - shortcut: ts - disabled: true - - - name: tmdb - engine: xpath - paging: true - categories: movies - search_url: https://www.themoviedb.org/search?page={pageno}&query={query} - results_xpath: //div[contains(@class,"movie") or contains(@class,"tv")]//div[contains(@class,"card")] - url_xpath: .//div[contains(@class,"poster")]/a/@href - thumbnail_xpath: .//img/@src - title_xpath: .//div[contains(@class,"title")]//h2 - content_xpath: .//div[contains(@class,"overview")] - shortcut: tm - disabled: true - - # Requires Tor - - name: torch - engine: xpath - paging: true - search_url: - http://xmh57jrknzkhv6y3ls3ubitzfqnkrwxhopf5aygthi7d6rplyvk3noyd.onion/cgi-bin/omega/omega?P={query}&DEFAULTOP=and - results_xpath: //table//tr - url_xpath: ./td[2]/a - title_xpath: ./td[2]/b - content_xpath: ./td[2]/small - categories: onions - enable_http: true - shortcut: tch - - # torznab engine lets you query any torznab compatible indexer. Using this - # engine in combination with Jackett opens the possibility to query a lot of - # public and private indexers directly from SearXNG. More details at: - # https://docs.searxng.org/dev/engines/online/torznab.html - # - # - name: Torznab EZTV - # engine: torznab - # shortcut: eztv - # base_url: http://localhost:9117/api/v2.0/indexers/eztv/results/torznab - # enable_http: true # if using localhost - # api_key: xxxxxxxxxxxxxxx - # show_magnet_links: true - # show_torrent_files: false - # # https://github.com/Jackett/Jackett/wiki/Jackett-Categories - # torznab_categories: # optional - # - 2000 - # - 5000 - - # tmp suspended - too slow, too many errors - # - name: urbandictionary - # engine : xpath - # search_url : https://www.urbandictionary.com/define.php?term={query} - # url_xpath : //*[@class="word"]/@href - # title_xpath : //*[@class="def-header"] - # content_xpath: //*[@class="meaning"] - # shortcut: ud - - - name: unsplash - engine: unsplash - shortcut: us - - - name: yandex - engine: yandex - categories: general - search_type: web - shortcut: yd - disabled: true - inactive: true - - - name: yandex images - engine: yandex - categories: images - search_type: images - shortcut: ydi - disabled: true - inactive: true - - - name: yandex music - engine: yandex_music - shortcut: ydm - disabled: true - # https://yandex.com/support/music/access.html - inactive: true - - - name: yahoo - engine: yahoo - shortcut: yh - disabled: true - - - name: yahoo news - engine: yahoo_news - shortcut: yhn - - - name: youtube - shortcut: yt - # You can use the engine using the official stable API, but you need an API - # key See: https://console.developers.google.com/project - # - # engine: youtube_api - # api_key: 'apikey' # required! - # - # Or you can use the html non-stable engine, activated by default - engine: youtube_noapi - - - name: dailymotion - engine: dailymotion - shortcut: dm - - - name: vimeo - engine: vimeo - shortcut: vm - - - name: wiby - engine: json_engine - paging: true - search_url: https://wiby.me/json/?q={query}&p={pageno} - url_query: URL - title_query: Title - content_query: Snippet - categories: [general, web] - shortcut: wib - disabled: true - about: - website: https://wiby.me/ - - - name: wikibooks - engine: mediawiki - weight: 0.5 - shortcut: wb - categories: [general, wikimedia] - base_url: "https://{language}.wikibooks.org/" - search_type: text - disabled: true - about: - website: https://www.wikibooks.org/ - wikidata_id: Q367 - - - name: wikinews - engine: mediawiki - shortcut: wn - categories: [news, wikimedia] - base_url: "https://{language}.wikinews.org/" - search_type: text - srsort: create_timestamp_desc - about: - website: https://www.wikinews.org/ - wikidata_id: Q964 - - - name: wikiquote - engine: mediawiki - weight: 0.5 - shortcut: wq - categories: [general, wikimedia] - base_url: "https://{language}.wikiquote.org/" - search_type: text - disabled: true - additional_tests: - rosebud: *test_rosebud - about: - website: https://www.wikiquote.org/ - wikidata_id: Q369 - - - name: wikisource - engine: mediawiki - weight: 0.5 - shortcut: ws - categories: [general, wikimedia] - base_url: "https://{language}.wikisource.org/" - search_type: text - disabled: true - about: - website: https://www.wikisource.org/ - wikidata_id: Q263 - - - name: wikispecies - engine: mediawiki - shortcut: wsp - categories: [general, science, wikimedia] - base_url: "https://species.wikimedia.org/" - search_type: text - disabled: true - about: - website: https://species.wikimedia.org/ - wikidata_id: Q13679 - tests: - wikispecies: - matrix: - query: "Campbell, L.I. et al. 2011: MicroRNAs" - lang: en - result_container: - - not_empty - - ['one_title_contains', 'Tardigrada'] - test: - - unique_results - - - name: wiktionary - engine: mediawiki - shortcut: wt - categories: [dictionaries, wikimedia] - base_url: "https://{language}.wiktionary.org/" - search_type: text - about: - website: https://www.wiktionary.org/ - wikidata_id: Q151 - - - name: wikiversity - engine: mediawiki - weight: 0.5 - shortcut: wv - categories: [general, wikimedia] - base_url: "https://{language}.wikiversity.org/" - search_type: text - disabled: true - about: - website: https://www.wikiversity.org/ - wikidata_id: Q370 - - - name: wikivoyage - engine: mediawiki - weight: 0.5 - shortcut: wy - categories: [general, wikimedia] - base_url: "https://{language}.wikivoyage.org/" - search_type: text - disabled: true - about: - website: https://www.wikivoyage.org/ - wikidata_id: Q373 - - - name: wikicommons.images - engine: wikicommons - shortcut: wc - categories: images - search_type: images - number_of_results: 10 - - - name: wikicommons.videos - engine: wikicommons - shortcut: wcv - categories: videos - search_type: videos - number_of_results: 10 - - - name: wikicommons.audio - engine: wikicommons - shortcut: wca - categories: music - search_type: audio - number_of_results: 10 - - - name: wikicommons.files - engine: wikicommons - shortcut: wcf - categories: files - search_type: files - number_of_results: 10 - - - name: wolframalpha - shortcut: wa - # You can use the engine using the official stable API, but you need an API - # key. See: https://products.wolframalpha.com/api/ - # - # engine: wolframalpha_api - # api_key: '' - # - # Or you can use the html non-stable engine, activated by default - engine: wolframalpha_noapi - timeout: 6.0 - categories: general - disabled: true - - - name: dictzone - engine: dictzone - shortcut: dc - - - name: mymemory translated - engine: translated - shortcut: tl - timeout: 5.0 - # You can use without an API key, but you are limited to 1000 words/day - # See: https://mymemory.translated.net/doc/usagelimits.php - # api_key: '' - - # Required dependency: mysql-connector-python - # - name: mysql - # engine: mysql_server - # database: mydatabase - # username: user - # password: pass - # limit: 10 - # query_str: 'SELECT * from mytable WHERE fieldname=%(query)s' - # shortcut: mysql - - # Required dependency: mariadb - # - name: mariadb - # engine: mariadb_server - # database: mydatabase - # username: user - # password: pass - # limit: 10 - # query_str: 'SELECT * from mytable WHERE fieldname=%(query)s' - # shortcut: mdb - - - name: 1337x - engine: 1337x - shortcut: 1337x - disabled: true - - - name: duden - engine: duden - shortcut: du - disabled: true - - - name: seznam - shortcut: szn - engine: seznam - disabled: true - - # - name: deepl - # engine: deepl - # shortcut: dpl - # # You can use the engine using the official stable API, but you need an API key - # # See: https://www.deepl.com/pro-api?cta=header-pro-api - # api_key: '' # required! - # timeout: 5.0 - # disabled: true - - - name: mojeek - shortcut: mjk - engine: mojeek - categories: [general, web] - disabled: true - - - name: mojeek images - shortcut: mjkimg - engine: mojeek - categories: [images, web] - search_type: images - paging: false - disabled: true - - - name: mojeek news - shortcut: mjknews - engine: mojeek - categories: [news, web] - search_type: news - paging: false - disabled: true - - - name: moviepilot - engine: moviepilot - shortcut: mp - disabled: true - - - name: naver - shortcut: nvr - categories: [general, web] - engine: xpath - paging: true - search_url: https://search.naver.com/search.naver?where=webkr&sm=osp_hty&ie=UTF-8&query={query}&start={pageno} - url_xpath: //a[@class="link_tit"]/@href - title_xpath: //a[@class="link_tit"] - content_xpath: //div[@class="total_dsc_wrap"]/a - first_page_num: 1 - page_size: 10 - disabled: true - about: - website: https://www.naver.com/ - wikidata_id: Q485639 - official_api_documentation: https://developers.naver.com/docs/nmt/examples/ - use_official_api: false - require_api_key: false - results: HTML - language: ko - - - name: rubygems - shortcut: rbg - engine: xpath - paging: true - search_url: https://rubygems.org/search?page={pageno}&query={query} - results_xpath: /html/body/main/div/a[@class="gems__gem"] - url_xpath: ./@href - title_xpath: ./span/h2 - content_xpath: ./span/p - suggestion_xpath: /html/body/main/div/div[@class="search__suggestions"]/p/a - first_page_num: 1 - categories: [it, packages] - disabled: true - about: - website: https://rubygems.org/ - wikidata_id: Q1853420 - official_api_documentation: https://guides.rubygems.org/rubygems-org-api/ - use_official_api: false - require_api_key: false - results: HTML - - - name: peertube - engine: peertube - shortcut: ptb - paging: true - # alternatives see: https://instances.joinpeertube.org/instances - # base_url: https://tube.4aem.com - categories: videos - disabled: true - timeout: 6.0 - - - name: mediathekviewweb - engine: mediathekviewweb - shortcut: mvw - disabled: true - - - name: yacy - # https://docs.searxng.org/dev/engines/online/yacy.html - engine: yacy - categories: general - search_type: text - base_url: - - https://yacy.searchlab.eu - # see https://github.com/searxng/searxng/pull/3631#issuecomment-2240903027 - # - https://search.kyun.li - # - https://yacy.securecomcorp.eu - # - https://yacy.myserv.ca - # - https://yacy.nsupdate.info - # - https://yacy.electroncash.de - shortcut: ya - disabled: true - # if you aren't using HTTPS for your local yacy instance disable https - # enable_http: false - search_mode: 'global' - # timeout can be reduced in 'local' search mode - timeout: 5.0 - - - name: yacy images - engine: yacy - network: yacy - categories: images - search_type: image - shortcut: yai - disabled: true - # timeout can be reduced in 'local' search mode - timeout: 5.0 - - - name: rumble - engine: rumble - shortcut: ru - base_url: https://rumble.com/ - paging: true - categories: videos - disabled: true - - - name: livespace - engine: livespace - shortcut: ls - categories: videos - disabled: true - timeout: 5.0 - - - name: wordnik - engine: wordnik - shortcut: def - categories: [dictionaries] - timeout: 5.0 - - - name: woxikon.de synonyme - engine: xpath - shortcut: woxi - categories: [dictionaries] - timeout: 5.0 - disabled: true - search_url: https://synonyme.woxikon.de/synonyme/{query}.php - url_xpath: //div[@class="upper-synonyms"]/a/@href - content_xpath: //div[@class="synonyms-list-group"] - title_xpath: //div[@class="upper-synonyms"]/a - no_result_for_http_status: [404] - about: - website: https://www.woxikon.de/ - wikidata_id: # No Wikidata ID - use_official_api: false - require_api_key: false - results: HTML - language: de - - - name: seekr news - engine: seekr - shortcut: senews - categories: news - seekr_category: news - disabled: true - - - name: seekr images - engine: seekr - network: seekr news - shortcut: seimg - categories: images - seekr_category: images - disabled: true - - - name: seekr videos - engine: seekr - network: seekr news - shortcut: sevid - categories: videos - seekr_category: videos - disabled: true - - - name: stract - engine: stract - shortcut: str - disabled: true - - - name: svgrepo - engine: svgrepo - shortcut: svg - timeout: 10.0 - disabled: true - - - name: tootfinder - engine: tootfinder - shortcut: toot - - - name: voidlinux - engine: voidlinux - shortcut: void - disabled: true - - - name: wallhaven - engine: wallhaven - # api_key: abcdefghijklmnopqrstuvwxyz - shortcut: wh - - # wikimini: online encyclopedia for children - # The fulltext and title parameter is necessary for Wikimini because - # sometimes it will not show the results and redirect instead - - name: wikimini - engine: xpath - shortcut: wkmn - search_url: https://fr.wikimini.org/w/index.php?search={query}&title=Sp%C3%A9cial%3ASearch&fulltext=Search - url_xpath: //li/div[@class="mw-search-result-heading"]/a/@href - title_xpath: //li//div[@class="mw-search-result-heading"]/a - content_xpath: //li/div[@class="searchresult"] - categories: general - disabled: true - about: - website: https://wikimini.org/ - wikidata_id: Q3568032 - use_official_api: false - require_api_key: false - results: HTML - language: fr - - - name: wttr.in - engine: wttr - shortcut: wttr - timeout: 9.0 - - - name: yummly - engine: yummly - shortcut: yum - disabled: true - - - name: brave - engine: brave - shortcut: br - time_range_support: true - paging: true - categories: [general, web] - brave_category: search - # brave_spellcheck: true - - - name: brave.images - engine: brave - network: brave - shortcut: brimg - categories: [images, web] - brave_category: images - - - name: brave.videos - engine: brave - network: brave - shortcut: brvid - categories: [videos, web] - brave_category: videos - - - name: brave.news - engine: brave - network: brave - shortcut: brnews - categories: news - brave_category: news - - # - name: brave.goggles - # engine: brave - # network: brave - # shortcut: brgog - # time_range_support: true - # paging: true - # categories: [general, web] - # brave_category: goggles - # Goggles: # required! This should be a URL ending in .goggle - - - name: lib.rs - shortcut: lrs - engine: lib_rs - disabled: true - - - name: sourcehut - shortcut: srht - engine: xpath - paging: true - search_url: https://sr.ht/projects?page={pageno}&search={query} - results_xpath: (//div[@class="event-list"])[1]/div[@class="event"] - url_xpath: ./h4/a[2]/@href - title_xpath: ./h4/a[2] - content_xpath: ./p - first_page_num: 1 - categories: [it, repos] - disabled: true - about: - website: https://sr.ht - wikidata_id: Q78514485 - official_api_documentation: https://man.sr.ht/ - use_official_api: false - require_api_key: false - results: HTML - - - name: goo - shortcut: goo - engine: xpath - paging: true - search_url: https://search.goo.ne.jp/web.jsp?MT={query}&FR={pageno}0 - url_xpath: //div[@class="result"]/p[@class='title fsL1']/a/@href - title_xpath: //div[@class="result"]/p[@class='title fsL1']/a - content_xpath: //p[contains(@class,'url fsM')]/following-sibling::p - first_page_num: 0 - categories: [general, web] - disabled: true - timeout: 4.0 - about: - website: https://search.goo.ne.jp - wikidata_id: Q249044 - use_official_api: false - require_api_key: false - results: HTML - language: ja - - - name: bt4g - engine: bt4g - shortcut: bt4g - - - name: pkg.go.dev - engine: pkg_go_dev - shortcut: pgo - disabled: true - - - name: senscritique - engine: senscritique - shortcut: scr - timeout: 4.0 - disabled: true - -# Doku engine lets you access to any Doku wiki instance: -# A public one or a privete/corporate one. -# - name: ubuntuwiki -# engine: doku -# shortcut: uw -# base_url: 'https://doc.ubuntu-fr.org' - -# Be careful when enabling this engine if you are -# running a public instance. Do not expose any sensitive -# information. You can restrict access by configuring a list -# of access tokens under tokens. -# - name: git grep -# engine: command -# command: ['git', 'grep', '{{QUERY}}'] -# shortcut: gg -# tokens: [] -# disabled: true -# delimiter: -# chars: ':' -# keys: ['filepath', 'code'] - -# Be careful when enabling this engine if you are -# running a public instance. Do not expose any sensitive -# information. You can restrict access by configuring a list -# of access tokens under tokens. -# - name: locate -# engine: command -# command: ['locate', '{{QUERY}}'] -# shortcut: loc -# tokens: [] -# disabled: true -# delimiter: -# chars: ' ' -# keys: ['line'] - -# Be careful when enabling this engine if you are -# running a public instance. Do not expose any sensitive -# information. You can restrict access by configuring a list -# of access tokens under tokens. -# - name: find -# engine: command -# command: ['find', '.', '-name', '{{QUERY}}'] -# query_type: path -# shortcut: fnd -# tokens: [] -# disabled: true -# delimiter: -# chars: ' ' -# keys: ['line'] - -# Be careful when enabling this engine if you are -# running a public instance. Do not expose any sensitive -# information. You can restrict access by configuring a list -# of access tokens under tokens. -# - name: pattern search in files -# engine: command -# command: ['fgrep', '{{QUERY}}'] -# shortcut: fgr -# tokens: [] -# disabled: true -# delimiter: -# chars: ' ' -# keys: ['line'] - -# Be careful when enabling this engine if you are -# running a public instance. Do not expose any sensitive -# information. You can restrict access by configuring a list -# of access tokens under tokens. -# - name: regex search in files -# engine: command -# command: ['grep', '{{QUERY}}'] -# shortcut: gr -# tokens: [] -# disabled: true -# delimiter: -# chars: ' ' -# keys: ['line'] - -doi_resolvers: - oadoi.org: 'https://oadoi.org/' - doi.org: 'https://doi.org/' - doai.io: 'https://dissem.in/' - sci-hub.se: 'https://sci-hub.se/' - sci-hub.st: 'https://sci-hub.st/' - sci-hub.ru: 'https://sci-hub.ru/' - -default_doi_resolver: 'oadoi.org' diff --git a/searxng/uwsgi.ini.new b/searxng/uwsgi.ini.new deleted file mode 100644 index c3860b3..0000000 --- a/searxng/uwsgi.ini.new +++ /dev/null @@ -1,55 +0,0 @@ -[uwsgi] -# Listening address -# default value: [::]:8080 (see Dockerfile) -http-socket = $(BIND_ADDRESS) - -# Who will run the code -uid = searxng -gid = searxng - -# Number of workers (usually CPU count) -# default value: %k (= number of CPU core, see Dockerfile) -workers = 4 - -# Number of threads per worker -# default value: 4 (see Dockerfile) -threads = 4 - -# The right granted on the created socket -chmod-socket = 666 - -# Plugin to use and interpreter config -single-interpreter = true -master = true -lazy-apps = true -enable-threads = 4 - -# Module to import -module = searx.webapp - -# Virtualenv and python path -pythonpath = /usr/local/searxng/ -chdir = /usr/local/searxng/searx/ - -# automatically set processes name to something meaningful -auto-procname = true - -# Disable request logging for privacy -disable-logging = true -log-5xx = true - -# Set the max size of a request (request-body excluded) -buffer-size = 8192 - -# No keep alive -# See https://github.com/searx/searx-docker/issues/24 -add-header = Connection: close - -# Follow SIGTERM convention -# See https://github.com/searxng/searxng/issues/3427 -die-on-term - -# uwsgi serves the static files -static-map = /static=/usr/local/searxng/searx/static -static-gzip-all = True -offload-threads = 4 From 9a34ff36465f25b92d1835334c17aad3f648c03a Mon Sep 17 00:00:00 2001 From: martin legrand Date: Tue, 27 May 2025 19:30:26 +0200 Subject: [PATCH 05/14] set config.ini back like before --- config.ini | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/config.ini b/config.ini index e71feeb..bfd9240 100644 --- a/config.ini +++ b/config.ini @@ -1,16 +1,15 @@ [MAIN] -is_local = False -provider_name = together -provider_model = perplexity-ai/r1-1776 -provider_server_address = 127.0.0.1:5000 -agent_name = Jarvis +is_local = True +provider_name = ollama +provider_model = deepseek-r1:14b +provider_server_address = 127.0.0.1:11434 +agent_name = Name_of_your_AI recover_last_session = False save_session = False -speak = True +speak = False listen = False -work_dir = ${WORK_DIR} -jarvis_personality = True +work_dir = /Users/mlg/Documents/workspace_for_agenticseek +jarvis_personality = False languages = en [BROWSER] -headless_browser = True -stealth_mode = False \ No newline at end of file +headless_browser = True \ No newline at end of file From 97460ded4815f9455e2a0fce6144f6b07ed54217 Mon Sep 17 00:00:00 2001 From: martin legrand Date: Tue, 27 May 2025 19:31:28 +0200 Subject: [PATCH 06/14] set config.ini back like before --- config.ini | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/config.ini b/config.ini index bfd9240..19cb1e4 100644 --- a/config.ini +++ b/config.ini @@ -12,4 +12,5 @@ work_dir = /Users/mlg/Documents/workspace_for_agenticseek jarvis_personality = False languages = en [BROWSER] -headless_browser = True \ No newline at end of file +headless_browser = True +stealth_mode = False \ No newline at end of file From c8df9e759c4d698d203a724a9200317fd08a51e7 Mon Sep 17 00:00:00 2001 From: lck Date: Wed, 28 May 2025 02:26:41 +0800 Subject: [PATCH 07/14] fix: handle missing tag in remove_reasoning_text --- sources/agents/agent.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sources/agents/agent.py b/sources/agents/agent.py index 0e6bfaf..15d6ae0 100644 --- a/sources/agents/agent.py +++ b/sources/agents/agent.py @@ -140,8 +140,10 @@ class Agent(): Remove the reasoning block of reasoning model like deepseek. """ end_tag = "" - end_idx = text.rfind(end_tag)+8 - return text[end_idx:] + end_idx = text.rfind(end_tag) + if end_idx == -1: + return text + return text[end_idx+8:] def extract_reasoning_text(self, text: str) -> None: """ From c8bccc2395d55d4bd3f2b2bc997b1a9b3da7014f Mon Sep 17 00:00:00 2001 From: Hung Nguyen Date: Wed, 28 May 2025 11:44:00 +1000 Subject: [PATCH 08/14] Added unit tests for tools parsing --- tests/test_tools_parsing.py | 237 ++++++++++++++++++++++++++++++++++++ 1 file changed, 237 insertions(+) create mode 100644 tests/test_tools_parsing.py diff --git a/tests/test_tools_parsing.py b/tests/test_tools_parsing.py new file mode 100644 index 0000000..8b06f32 --- /dev/null +++ b/tests/test_tools_parsing.py @@ -0,0 +1,237 @@ +import unittest +import os +import sys +sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) # Add project root to Python path + +from sources.tools.tools import Tools + +class TestToolsParsing(unittest.TestCase): + """ + Test suite for the Tools class parsing functionality, specifically the load_exec_block method. + This method is responsible for extracting code blocks from LLM-generated text. + """ + + def setUp(self): + """Set up test fixtures before each test method.""" + # Create a concrete implementation of the abstract Tools class for testing + class TestTool(Tools): + def execute(self, blocks, safety=False): + return "test execution" + + def execution_failure_check(self, output): + return False + + def interpreter_feedback(self, output): + return "test feedback" + + self.tool = TestTool() + self.tool.tag = "python" # Set tag for testing + + def test_load_exec_block_single_block(self): + """Test parsing a single code block from LLM text.""" + llm_text = """Here's some Python code: +```python +print("Hello, World!") +x = 42 +``` +That's the code.""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 1) + self.assertEqual(blocks[0], '\nprint("Hello, World!")\nx = 42\n') + self.assertIsNone(save_path) + + def test_load_exec_block_multiple_blocks(self): + """Test parsing multiple code blocks from LLM text.""" + llm_text = """First block: +```python +import os +print("First block") +``` + +Second block: +```python +import sys +print("Second block") +``` + +Done.""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 2) + self.assertEqual(blocks[0], '\nimport os\nprint("First block")\n') + self.assertEqual(blocks[1], '\nimport sys\nprint("Second block")\n') + self.assertIsNone(save_path) + + def test_load_exec_block_with_save_path(self): + """Test parsing code block with save path specification.""" + llm_text = """```python +save_path: test_file.py +import os +print("Hello with save path") +```""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 1) + # The save path logic only works when the first line (after newline) contains a colon + # In this case, the first line is empty, so save_path parsing doesn't trigger + self.assertEqual(blocks[0], '\nsave_path: test_file.py\nimport os\nprint("Hello with save path")\n') + self.assertIsNone(save_path) + + + def test_load_exec_block_with_indentation(self): + """Test parsing code blocks with leading whitespace/indentation.""" + llm_text = """ Here's indented code: + ```python + def hello(): + print("Hello") + return True + ``` + End of code.""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 1) + expected_code = '\ndef hello():\n print("Hello")\n return True\n' + self.assertEqual(blocks[0], expected_code) + + def test_load_exec_block_no_blocks(self): + """Test parsing text with no code blocks.""" + llm_text = """This is just regular text with no code blocks. +There are no python blocks here. +Just plain text.""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNone(blocks) + self.assertIsNone(save_path) + + def test_load_exec_block_wrong_tag(self): + """Test parsing text with code blocks but wrong language tag.""" + llm_text = """```javascript +console.log("This is JavaScript, not Python"); +```""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNone(blocks) + self.assertIsNone(save_path) + + def test_load_exec_block_incomplete_block(self): + """Test parsing text with incomplete code block (missing closing tag).""" + llm_text = """```python +print("This block has no closing tag") +x = 42""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + # Function returns empty list and None for incomplete blocks + self.assertEqual(blocks, []) + self.assertIsNone(save_path) + + def test_load_exec_block_empty_block(self): + """Test parsing empty code block.""" + llm_text = """```python +```""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 1) + self.assertEqual(blocks[0], '\n') + + def test_load_exec_block_mixed_content(self): + """Test parsing text with mixed content including code blocks.""" + llm_text = """Let me help you with that task. + +First, I'll import the necessary modules: +```python +import os +import sys +``` + +Then I'll define a function: +```python +def process_data(data): + return data.upper() +``` + +Finally, let's use it: +```python +result = process_data("hello world") +print(result) +``` + +That should work!""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 3) + self.assertEqual(blocks[0], '\nimport os\nimport sys\n') + self.assertEqual(blocks[1], '\ndef process_data(data):\n return data.upper()\n') + self.assertEqual(blocks[2], '\nresult = process_data("hello world")\nprint(result)\n') + + def test_load_exec_block_with_special_characters(self): + """Test parsing code blocks containing special characters.""" + llm_text = """```python +text = "Hello \"world\" with 'quotes'" +regex = r"^\\d+$" +path = "C:\\Users\\test\\file.txt" +```""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + self.assertIsNotNone(blocks) + self.assertEqual(len(blocks), 1) + expected = '\ntext = "Hello "world" with \'quotes\'"\nregex = r"^\\d+$"\npath = "C:\\Users\\test\\file.txt"\n' + self.assertEqual(blocks[0], expected) + + def test_load_exec_block_tag_undefined(self): + """Test that assertion error is raised when tag is undefined.""" + self.tool.tag = "undefined" + llm_text = """```python +print("test") +```""" + + with self.assertRaises(AssertionError): + self.tool.load_exec_block(llm_text) + + def test_found_executable_blocks_flag(self): + """Test that the executable blocks found flag is set correctly.""" + # Initially should be False + self.assertFalse(self.tool.found_executable_blocks()) + + llm_text = """```python +print("test") +```""" + + blocks, save_path = self.tool.load_exec_block(llm_text) + + # After finding blocks, should be True + self.assertTrue(self.tool.found_executable_blocks()) + + # After calling found_executable_blocks(), should reset to False + self.assertFalse(self.tool.found_executable_blocks()) + + def test_get_parameter_value(self): + """Test the get_parameter_value helper method.""" + block = """param1 = value1 +param2 = value2 +some other text +param3 = value3""" + + self.assertEqual(self.tool.get_parameter_value(block, "param1"), "value1") + self.assertEqual(self.tool.get_parameter_value(block, "param2"), "value2") + self.assertEqual(self.tool.get_parameter_value(block, "param3"), "value3") + self.assertIsNone(self.tool.get_parameter_value(block, "nonexistent")) + +if __name__ == '__main__': + unittest.main() \ No newline at end of file From 41fe95fcb1f0a44e019ebd9e96256fce9597838c Mon Sep 17 00:00:00 2001 From: Hung Nguyen Date: Wed, 28 May 2025 11:47:21 +1000 Subject: [PATCH 09/14] Refine test_tools_parsing --- tests/test_tools_parsing.py | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/tests/test_tools_parsing.py b/tests/test_tools_parsing.py index 8b06f32..5840f12 100644 --- a/tests/test_tools_parsing.py +++ b/tests/test_tools_parsing.py @@ -1,7 +1,7 @@ import unittest import os import sys -sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) # Add project root to Python path +sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) from sources.tools.tools import Tools @@ -13,7 +13,6 @@ class TestToolsParsing(unittest.TestCase): def setUp(self): """Set up test fixtures before each test method.""" - # Create a concrete implementation of the abstract Tools class for testing class TestTool(Tools): def execute(self, blocks, safety=False): return "test execution" @@ -79,8 +78,6 @@ print("Hello with save path") self.assertIsNotNone(blocks) self.assertEqual(len(blocks), 1) - # The save path logic only works when the first line (after newline) contains a colon - # In this case, the first line is empty, so save_path parsing doesn't trigger self.assertEqual(blocks[0], '\nsave_path: test_file.py\nimport os\nprint("Hello with save path")\n') self.assertIsNone(save_path) @@ -132,7 +129,6 @@ x = 42""" blocks, save_path = self.tool.load_exec_block(llm_text) - # Function returns empty list and None for incomplete blocks self.assertEqual(blocks, []) self.assertIsNone(save_path) @@ -206,7 +202,6 @@ print("test") def test_found_executable_blocks_flag(self): """Test that the executable blocks found flag is set correctly.""" - # Initially should be False self.assertFalse(self.tool.found_executable_blocks()) llm_text = """```python @@ -215,10 +210,8 @@ print("test") blocks, save_path = self.tool.load_exec_block(llm_text) - # After finding blocks, should be True self.assertTrue(self.tool.found_executable_blocks()) - # After calling found_executable_blocks(), should reset to False self.assertFalse(self.tool.found_executable_blocks()) def test_get_parameter_value(self): From a3ad635728163668cb73fdb62e608d3f5984bc04 Mon Sep 17 00:00:00 2001 From: martin legrand Date: Sun, 25 May 2025 21:19:18 +0200 Subject: [PATCH 10/14] deploy : current attempt at backend dockerization --- .env.example | 4 +- .gitignore | 1 + Dockerfile.backend | 89 +++++++++++++++++++------- README.md | 4 ++ api.py | 9 +++ docker-compose.yml | 21 +++++- frontend/agentic-seek-front/src/App.js | 12 ++-- requirements.txt | 1 + sources/browser.py | 14 +++- start_services.sh | 50 ++++++++++++++- 10 files changed, 170 insertions(+), 35 deletions(-) diff --git a/.env.example b/.env.example index 069f23c..0e98844 100644 --- a/.env.example +++ b/.env.example @@ -1,4 +1,6 @@ SEARXNG_BASE_URL="http://127.0.0.1:8080" OPENAI_API_KEY='xxxxx' DEEPSEEK_API_KEY='xxxxx' -OPENROUTER_API_KEY='xxxxx' \ No newline at end of file +OPENROUTER_API_KEY='xxxxx' +BACKEND_PORT=8000 +WORK_DIR="/tmp/" \ No newline at end of file diff --git a/.gitignore b/.gitignore index 0e82296..f8801e4 100644 --- a/.gitignore +++ b/.gitignore @@ -19,6 +19,7 @@ agentic_seek_env/* .env */.env dsk/ +chrome136/ ### react ### .DS_* diff --git a/Dockerfile.backend b/Dockerfile.backend index 1cb8149..9bae45f 100644 --- a/Dockerfile.backend +++ b/Dockerfile.backend @@ -1,38 +1,57 @@ FROM ubuntu:22.04 -# Warning: doesn't work yet, backend is run on host machine for now WORKDIR /app -RUN apt-get update -qq -y && \ -apt-get install -y \ - gcc \ - g++ \ - gfortran \ - libportaudio2 \ - portaudio19-dev \ - ffmpeg \ - libavcodec-dev \ - libavformat-dev \ - libavutil-dev \ - gnupg2 \ - wget \ - unzip \ - python3 \ - python3-pip \ - libasound2 \ - libatk-bridge2.0-0 \ - libgtk-4-1 \ - libnss3 \ - xdg-utils \ - wget && \ +# Install essential packages and Chrome dependencies +RUN apt-get update && apt-get install -y \ + wget \ + unzip \ + curl \ + gnupg \ + python3-dev \ + python3-pip \ + python3-wheel \ + build-essential \ + # Chrome dependencies - comprehensive list + fonts-liberation \ + libasound2 \ + libatk-bridge2.0-0 \ + libdrm2 \ + libxcomposite1 \ + libxdamage1 \ + libxrandr2 \ + libgbm1 \ + libxss1 \ + libnss3 \ + libnspr4 \ + libxshmfence1 \ + libgconf-2-4 \ + libxfixes3 \ + libxinerama1 \ + libgtk-3-0 \ + libgdk-pixbuf2.0-0 \ + libatspi2.0-0 \ + libdrm2 \ + libxkbcommon0 \ + libepoxy0 \ + libgtk-3-0 \ + libharfbuzz0b \ + libegl1-mesa \ + libgles2-mesa \ + # Virtual display for headless operation + xvfb \ + && rm -rf /var/lib/apt/lists/* + +RUN pip3 install --upgrade pip setuptools wheel && \ +pip3 install selenium -RUN chmod +x /opt/chrome/chrome # Install dependencies COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt # Copy application code COPY api.py . +COPY chrome_bundle/ ./chrome_bundle/ COPY sources/ ./sources/ COPY prompts/ ./prompts/ COPY crx/ crx/ @@ -40,6 +59,28 @@ COPY llm_router/ llm_router/ COPY .env . COPY config.ini . +# Install Chrome and ChromeDriver from chrome-for-testing +RUN wget -O chrome-linux64.zip https://storage.googleapis.com/chrome-for-testing-public/136.0.7103.113/linux64/chrome-linux64.zip && \ + wget -O chromedriver-linux64.zip https://storage.googleapis.com/chrome-for-testing-public/136.0.7103.113/linux64/chromedriver-linux64.zip && \ + unzip chrome-linux64.zip && \ + unzip chromedriver-linux64.zip && \ + mkdir -p /opt/google && \ + ls -la && \ + mv chrome-linux64 /opt/google/chrome && \ + mv chromedriver-linux64/chromedriver /usr/local/bin/chromedriver && \ + chmod +x /opt/google/chrome/chrome && \ + chmod +x /usr/local/bin/chromedriver + +RUN ln -s /opt/google/chrome/chrome /usr/local/bin/chrome + +# Verify Chrome and ChromeDriver installation +RUN google-chrome --version +RUN chromedriver --version > /dev/null 2>&1 + +ENV CHROME_BIN=/opt/google/chrome/chrome +ENV CHROMEDRIVER_PATH=/usr/local/bin/chromedriver +ENV DISPLAY=:99 + # Expose port EXPOSE 8000 diff --git a/README.md b/README.md index 34612c5..ecf2486 100644 --- a/README.md +++ b/README.md @@ -559,3 +559,7 @@ We’re looking for developers to improve AgenticSeek! Check out open issues or > [antoineVIVIES](https://github.com/antoineVIVIES) | Taipei Time > [steveh8758](https://github.com/steveh8758) | Taipei Time + +## Special Thanks: + + > [tcsenpai](https://github.com/tcsenpai) For dockerization of backend diff --git a/api.py b/api.py index fa82896..3689c04 100755 --- a/api.py +++ b/api.py @@ -22,6 +22,10 @@ from sources.utility import pretty_print from sources.logger import Logger from sources.schemas import QueryRequest, QueryResponse +from dotenv import load_dotenv + +load_dotenv() + from celery import Celery @@ -247,4 +251,9 @@ async def process_query(request: QueryRequest): interaction.save_session() if __name__ == "__main__": + envport = os.getenv("BACKEND_PORT") + if envport: + port = int(envport) + else: + port = 8000 uvicorn.run(api, host="0.0.0.0", port=8000) \ No newline at end of file diff --git a/docker-compose.yml b/docker-compose.yml index 2f78cb9..f2a0d6c 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -31,7 +31,7 @@ services: volumes: - ./searxng:/etc/searxng:rw environment: - - SEARXNG_BASE_URL=http://localhost:8080/ + - SEARXNG_BASE_URL=${SEARXNG_BASE_URL:-http://localhost:8080/} - SEARXNG_SECRET_KEY=$(openssl rand -hex 32) - UWSGI_WORKERS=4 - UWSGI_THREADS=4 @@ -62,13 +62,28 @@ services: environment: - NODE_ENV=development - CHOKIDAR_USEPOLLING=true - - BACKEND_URL=http://backend:8000 + - REACT_APP_BACKEND_URL=http://0.0.0.0:${BACKEND_PORT:-8000} networks: - agentic-seek-net + + backend: + container_name: backend + build: + context: . + dockerfile: Dockerfile.backend + ports: + - ${BACKEND_PORT:-8000}:${BACKEND_PORT:-8000} + volumes: + - ./:/app + - ${WORK_DIR}:${WORK_DIR} + command: python3 api.py + environment: + - SEARXNG_URL=http://localhost:8080 + - WORK_DIR=${WORK_DIR} + network_mode: "host" # NOTE: backend service is not working yet due to issue with chromedriver on docker. # Therefore backend is run on host machine. - # Open to pull requests to fix this. #backend: # container_name: backend diff --git a/frontend/agentic-seek-front/src/App.js b/frontend/agentic-seek-front/src/App.js index c1e6ba8..a9e6407 100644 --- a/frontend/agentic-seek-front/src/App.js +++ b/frontend/agentic-seek-front/src/App.js @@ -4,6 +4,8 @@ import axios from 'axios'; import './App.css'; import { colors } from './colors'; +const BACKEND_URL = process.env.REACT_APP_BACKEND_URL || 'http://0.0.0.0:8000'; + function App() { const [query, setQuery] = useState(''); const [messages, setMessages] = useState([]); @@ -27,7 +29,7 @@ function App() { const checkHealth = async () => { try { - await axios.get('http://127.0.0.1:8000/health'); + await axios.get(`${BACKEND_URL}/health`); setIsOnline(true); console.log('System is online'); } catch { @@ -39,7 +41,7 @@ function App() { const fetchScreenshot = async () => { try { const timestamp = new Date().getTime(); - const res = await axios.get(`http://127.0.0.1:8000/screenshots/updated_screen.png?timestamp=${timestamp}`, { + const res = await axios.get(`${BACKEND_URL}/screenshots/updated_screen.png?timestamp=${timestamp}`, { responseType: 'blob' }); console.log('Screenshot fetched successfully'); @@ -90,7 +92,7 @@ function App() { const fetchLatestAnswer = async () => { try { - const res = await axios.get('http://127.0.0.1:8000/latest_answer'); + const res = await axios.get(`${BACKEND_URL}/latest_answer`); const data = res.data; updateData(data); @@ -141,7 +143,7 @@ function App() { setIsLoading(false); setError(null); try { - const res = await axios.get('http://127.0.0.1:8000/stop'); + const res = await axios.get(`${BACKEND_URL}/stop`); setStatus("Requesting stop..."); } catch (err) { console.error('Error stopping the agent:', err); @@ -162,7 +164,7 @@ function App() { try { console.log('Sending query:', query); setQuery('waiting for response...'); - const res = await axios.post('http://127.0.0.1:8000/query', { + const res = await axios.post(`${BACKEND_URL}/query`, { query, tts_enabled: false }); diff --git a/requirements.txt b/requirements.txt index a63646c..eba75e1 100644 --- a/requirements.txt +++ b/requirements.txt @@ -41,6 +41,7 @@ fake_useragent>=2.1.0 selenium_stealth>=1.0.6 undetected-chromedriver>=3.5.5 sentencepiece>=0.2.0 +python-dotenv>=1.0.0 tqdm>4 openai sniffio diff --git a/sources/browser.py b/sources/browser.py index 3604e9b..f3ebad4 100644 --- a/sources/browser.py +++ b/sources/browser.py @@ -42,7 +42,14 @@ def get_chrome_path() -> str: paths = ["/Applications/Google Chrome.app/Contents/MacOS/Google Chrome", "/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta"] else: # Linux - paths = ["/usr/bin/google-chrome", "/usr/bin/chromium-browser", "/usr/bin/chromium", "/opt/chrome/chrome", "/usr/local/bin/chrome"] + paths = ["/usr/bin/google-chrome", + "/usr/bin/chromium-browser", + "/usr/bin/chromium", + "/opt/chrome/chrome", + "opt/google/chrome/chrome", + "/usr/local/bin/chrome", + #"/app/chrome_bundle/chrome136/chrome-linux64" + ] for path in paths: if os.path.exists(path) and os.access(path, os.X_OK): @@ -73,8 +80,13 @@ def install_chromedriver() -> str: Install the ChromeDriver if not already installed. Return the path. """ chromedriver_path = shutil.which("chromedriver") + #if not chromedriver_path: + # if os.path.exists("/app/chrome_bundle/chrome136/chromedriver"): + # print("Using bundled ChromeDriver from /app/chrome_bundle/chrome136/chromedriver") + # chromedriver_path = "/app/chrome_bundle/chrome136/chromedriver" if not chromedriver_path: try: + print("ChromeDriver not found, attempting to install automatically...") chromedriver_path = chromedriver_autoinstaller.install() except Exception as e: raise FileNotFoundError( diff --git a/start_services.sh b/start_services.sh index 995f6ff..32759e8 100755 --- a/start_services.sh +++ b/start_services.sh @@ -1,5 +1,7 @@ #!/bin/bash +source .env + command_exists() { command -v "$1" &> /dev/null } @@ -60,12 +62,58 @@ if [ ! -f "docker-compose.yml" ]; then exit 1 fi -# start docker compose for searxng, redis, frontend services +# Download and extract Chrome bundle if not present +echo "Checking Chrome bundle..." +if [ ! -d "chrome_bundle/chrome136" ]; then + echo "Chrome bundle not found. Downloading..." + mkdir -p chrome_bundle + curl -L https://github.com/tcsenpai/agenticSeek/releases/download/utility/chrome136.zip -o /tmp/chrome136.zip + if [ $? -ne 0 ]; then + echo "Error: Failed to download Chrome bundle" + exit 1 + fi + unzip -q /tmp/chrome136.zip -d chrome_bundle/ + if [ $? -ne 0 ]; then + echo "Error: Failed to extract Chrome bundle" + exit 1 + fi + rm /tmp/chrome136.zip + echo "Chrome bundle downloaded and extracted successfully" +else + echo "Chrome bundle already exists" +fi + +# Stop all running containers to ensure a clean state echo "Warning: stopping all docker containers (t-4 seconds)..." sleep 4 docker stop $(docker ps -a -q) echo "All containers stopped" +# First start backend and wait for it to be healthy +echo "Starting backend service..." +if ! $COMPOSE_CMD up -d backend; then + echo "Error: Failed to start backend container." + exit 1 +fi + +# Wait for backend to be healthy (check if it's running and not restarting) +echo "Waiting for backend to be ready..." +for i in {1..30}; do + if [ "$(docker inspect -f '{{.State.Running}}' backend)" = "true" ] && \ + [ "$(docker inspect -f '{{.State.Restarting}}' backend)" = "false" ]; then + echo "backend is ready!" + break + fi + if [ $i -eq 30 ]; then + echo "Error: backend failed to start properly after 30 seconds" + $COMPOSE_CMD logs backend + exit 1 + fi + sleep 1 +done + +# start remaining services for searxng, redis, frontend services + if ! $COMPOSE_CMD up; then echo "Error: Failed to start containers. Check Docker logs with '$COMPOSE_CMD logs'." echo "Possible fixes: Run with sudo or ensure port 8080 is free." From 7d74a348c979d52feaa276d2556ddcc14d605387 Mon Sep 17 00:00:00 2001 From: martin legrand Date: Sun, 25 May 2025 21:56:29 +0200 Subject: [PATCH 11/14] comment out bundle approach --- Dockerfile.backend | 3 ++- start_services.sh | 39 ++++++++++++++++++++------------------- 2 files changed, 22 insertions(+), 20 deletions(-) diff --git a/Dockerfile.backend b/Dockerfile.backend index 9bae45f..0e9b2e9 100644 --- a/Dockerfile.backend +++ b/Dockerfile.backend @@ -51,7 +51,8 @@ RUN pip install --no-cache-dir -r requirements.txt # Copy application code COPY api.py . -COPY chrome_bundle/ ./chrome_bundle/ +# Chrome bundle approach, commented out opting for direct download +#COPY chrome_bundle/ ./chrome_bundle/ COPY sources/ ./sources/ COPY prompts/ ./prompts/ COPY crx/ crx/ diff --git a/start_services.sh b/start_services.sh index 32759e8..ef95319 100755 --- a/start_services.sh +++ b/start_services.sh @@ -62,26 +62,27 @@ if [ ! -f "docker-compose.yml" ]; then exit 1 fi +# bundle based approach, commented out in favor of direct download for now # Download and extract Chrome bundle if not present -echo "Checking Chrome bundle..." -if [ ! -d "chrome_bundle/chrome136" ]; then - echo "Chrome bundle not found. Downloading..." - mkdir -p chrome_bundle - curl -L https://github.com/tcsenpai/agenticSeek/releases/download/utility/chrome136.zip -o /tmp/chrome136.zip - if [ $? -ne 0 ]; then - echo "Error: Failed to download Chrome bundle" - exit 1 - fi - unzip -q /tmp/chrome136.zip -d chrome_bundle/ - if [ $? -ne 0 ]; then - echo "Error: Failed to extract Chrome bundle" - exit 1 - fi - rm /tmp/chrome136.zip - echo "Chrome bundle downloaded and extracted successfully" -else - echo "Chrome bundle already exists" -fi +#echo "Checking Chrome bundle..." +#if [ ! -d "chrome_bundle/chrome136" ]; then +# echo "Chrome bundle not found. Downloading..." +# mkdir -p chrome_bundle +# curl -L https://github.com/Fosowl/agenticSeek/releases/download/utility/chrome136.zip -o /tmp/chrome136.zip +# if [ $? -ne 0 ]; then +# echo "Error: Failed to download Chrome bundle" +# exit 1 +# fi +# unzip -q /tmp/chrome136.zip -d chrome_bundle/ +# if [ $? -ne 0 ]; then +# echo "Error: Failed to extract Chrome bundle" +# exit 1 +# fi +# rm /tmp/chrome136.zip +# echo "Chrome bundle downloaded and extracted successfully" +#else +# echo "Chrome bundle already exists" +#fi # Stop all running containers to ensure a clean state echo "Warning: stopping all docker containers (t-4 seconds)..." From 819a3fb98da7209b6140bbe38e69189153e1937b Mon Sep 17 00:00:00 2001 From: martin legrand Date: Sun, 25 May 2025 21:57:56 +0200 Subject: [PATCH 12/14] remove commented service --- docker-compose.yml | 29 ----------------------------- 1 file changed, 29 deletions(-) diff --git a/docker-compose.yml b/docker-compose.yml index f2a0d6c..df34535 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -82,35 +82,6 @@ services: - WORK_DIR=${WORK_DIR} network_mode: "host" - # NOTE: backend service is not working yet due to issue with chromedriver on docker. - # Therefore backend is run on host machine. - - #backend: - # container_name: backend - # build: - # context: ./ - # dockerfile: Dockerfile.backend - # stdin_open: true - # tty: true - # shm_size: 8g - # ports: - # - "8000:8000" - # volumes: - # - ./:/app - # environment: - # - NODE_ENV=development - # - REDIS_URL=redis://redis:6379/0 - # - SEARXNG_URL=http://searxng:8080 - # - OLLAMA_URL=http://localhost:11434 - # - LM_STUDIO_URL=http://localhost:1234 - # extra_hosts: - # - "host.docker.internal:host-gateway" - # depends_on: - # - redis - # - searxng - # networks: - # - agentic-seek-net - volumes: redis-data: chrome_profiles: From abae98cf77859e39ef0d8c0ee546efd9e37fb9f4 Mon Sep 17 00:00:00 2001 From: martin legrand Date: Sun, 25 May 2025 22:34:13 +0200 Subject: [PATCH 13/14] feat : optional run backend on host for start_services.sh --- docker-compose.yml | 4 ++++ start_services.sh | 56 +++++++++++++++++++++++++--------------------- 2 files changed, 34 insertions(+), 26 deletions(-) diff --git a/docker-compose.yml b/docker-compose.yml index df34535..824d869 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -3,6 +3,7 @@ version: '3' services: redis: container_name: redis + profiles: ["core", "full"] image: docker.io/valkey/valkey:8-alpine command: valkey-server --save 30 1 --loglevel warning restart: unless-stopped @@ -24,6 +25,7 @@ services: searxng: container_name: searxng + profiles: ["core", "full"] image: docker.io/searxng/searxng:latest restart: unless-stopped ports: @@ -51,6 +53,7 @@ services: frontend: container_name: frontend + profiles: ["core", "full"] build: context: ./frontend dockerfile: Dockerfile.frontend @@ -68,6 +71,7 @@ services: backend: container_name: backend + profiles: ["backend", "full"] build: context: . dockerfile: Dockerfile.backend diff --git a/start_services.sh b/start_services.sh index ef95319..a972ce2 100755 --- a/start_services.sh +++ b/start_services.sh @@ -90,34 +90,38 @@ sleep 4 docker stop $(docker ps -a -q) echo "All containers stopped" -# First start backend and wait for it to be healthy -echo "Starting backend service..." -if ! $COMPOSE_CMD up -d backend; then - echo "Error: Failed to start backend container." - exit 1 -fi - -# Wait for backend to be healthy (check if it's running and not restarting) -echo "Waiting for backend to be ready..." -for i in {1..30}; do - if [ "$(docker inspect -f '{{.State.Running}}' backend)" = "true" ] && \ - [ "$(docker inspect -f '{{.State.Restarting}}' backend)" = "false" ]; then - echo "backend is ready!" - break +if [ "$1" = "full" ]; then + # First start backend and wait for it to be healthy + echo "Full docker deployement. Starting backend service..." + if ! $COMPOSE_CMD up -d backend; then + echo "Error: Failed to start backend container." + exit 1 fi - if [ $i -eq 30 ]; then - echo "Error: backend failed to start properly after 30 seconds" - $COMPOSE_CMD logs backend + # Wait for backend to be healthy (check if it's running and not restarting) + echo "Waiting for backend to be ready..." + for i in {1..30}; do + if [ "$(docker inspect -f '{{.State.Running}}' backend)" = "true" ] && \ + [ "$(docker inspect -f '{{.State.Restarting}}' backend)" = "false" ]; then + echo "backend is ready!" + break + fi + if [ $i -eq 30 ]; then + echo "Error: backend failed to start properly after 30 seconds" + $COMPOSE_CMD logs backend + exit 1 + fi + sleep 1 + done + if ! $COMPOSE_CMD --profile full up; then + echo "Error: Failed to start containers. Check Docker logs with '$COMPOSE_CMD logs'." + echo "Possible fixes: Run with sudo or ensure port 8080 is free." + exit 1 + fi +else + if ! $COMPOSE_CMD --profile core up; then + echo "Error: Failed to start containers. Check Docker logs with '$COMPOSE_CMD logs'." + echo "Possible fixes: Run with sudo or ensure port 8080 is free." exit 1 fi - sleep 1 -done - -# start remaining services for searxng, redis, frontend services - -if ! $COMPOSE_CMD up; then - echo "Error: Failed to start containers. Check Docker logs with '$COMPOSE_CMD logs'." - echo "Possible fixes: Run with sudo or ensure port 8080 is free." - exit 1 fi sleep 10 \ No newline at end of file From ec1f7d31fbc357eb54b66076428a7d9e9a302325 Mon Sep 17 00:00:00 2001 From: martin legrand Date: Tue, 27 May 2025 18:19:20 +0200 Subject: [PATCH 14/14] update start_servicees.sh --- README.md | 2 +- start_services.sh | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ecf2486..cae9e18 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ Disclaimer: This demo, including all the files that appear (e.g: CV_candidates.z Make sure you have chrome driver, docker and python3.10 installed. -We highly advice you use exactly python3.10 for the setup. Dependencies error might happen otherwise. +We highly advise you use exactly python3.10 for the setup. Dependencies error might happen otherwise. For issues related to chrome driver, see the **Chromedriver** section. diff --git a/start_services.sh b/start_services.sh index a972ce2..07fcbb0 100755 --- a/start_services.sh +++ b/start_services.sh @@ -6,6 +6,12 @@ command_exists() { command -v "$1" &> /dev/null } +if [ "$1" = "full" ]; then + echo "Starting full deployment with backend and all services..." +else + echo "Starting core deployment with frontend and search services only..." +fi + # # Check if Docker is installed é running #