PyGPT - Desktop AI Assistant

Release: 2.7.12 | build: 2026-02-06 | Python: >=3.10, <3.14

Official website: https://pygpt.net | Documentation: https://pygpt.readthedocs.io

Discord: https://pygpt.net/discord | Snap: https://snapcraft.io/pygpt | PyPi: https://pypi.org/project/pygpt-net

Compiled version for Linux (zip) and Windows 10/11 (msi) 64-bit: https://pygpt.net/#download

❤️ Donate: https://www.buymeacoffee.com/szczyglis | https://github.com/sponsors/szczyglis-dev

Overview

PyGPT is all-in-one Desktop AI Assistant that provides direct interaction with OpenAI language models, including GPT-5, GPT-4, o1, o3 and more, through the OpenAI API. By utilizing other SDKs and LlamaIndex, the application also supports alternative LLMs, like those available on HuggingFace, locally available models via Ollama (like gpt-oss, Llama 3,Mistral, DeepSeek V3/R1 or Bielik), and other models like Google Gemini, Anthropic Claude, Perplexity / Sonar, and xAI Grok.

This assistant offers multiple modes of operation such as chat, assistants, agents, completions, and image-related tasks like image generation and image analysis. PyGPT has filesystem capabilities for file I/O, can generate and run Python code, execute system commands, execute custom commands and manage file transfers. It also allows models to perform web searches with the DuckDuckGo, Google and Microsoft Bing.

For audio interactions, PyGPT includes speech synthesis using the Microsoft Azure, Google, Eleven Labs and OpenAI Text-To-Speech services. Additionally, it features speech recognition capabilities provided by OpenAI Whisper, Google and Bing enabling the application to understand spoken commands and transcribe audio inputs into text. It features context memory with save and load functionality, enabling users to resume interactions from predefined points in the conversation. Prompt creation and management are streamlined through an intuitive preset system.

PyGPT's functionality extends through plugin support, allowing for custom enhancements (with multiple plugins included). Its multi-modal capabilities make it an adaptable tool for a range of AI-assisted operations, such as text-based interactions, system automation, daily assisting, vision applications, natural language processing, code generation and image creation.

Multiple operation modes are included, such as chat, text completion, assistant, agents, vision, Chat with Files (via LlamaIndex), commands execution, external API calls and image generation, making PyGPT a multi-tool for many AI-driven tasks.

Showcase (mp4, version 2.5.65, build 2025-07-24):

https://github.com/user-attachments/assets/d8305109-8b1b-41cb-b3ba-8c654271a95c

Screenshots (version 2.5.64, build 2025-07-23):

Dark theme:

Light theme:

You can download compiled 64-bit versions for Windows and Linux here: https://pygpt.net/#download

Features

• Desktop AI Assistant for Linux, Windows and Mac, written in Python.
• Works similarly to ChatGPT, but locally (on a desktop computer).
• 11 modes of operation: Chat, Chat with Files, Realtime + audio, Research (Perplexity), Completion, Image and Video generation, Assistants, Experts, Computer use, Agents and Autonomous Mode.
• Supports multiple models like OpenAI GPT-5, GPT-4, o1, o3, o4, Google Gemini, Anthropic Claude, xAI Grok, DeepSeek V3/R1, Perplexity / Sonar, and any model accessible through LlamaIndex and Ollama such as DeepSeek, gpt-oss, Llama 3, Mistral, Bielik, etc.
• Chat with your own Files: integrated LlamaIndex support: chat with data such as: txt, pdf, csv, html, md, docx, json, epub, xlsx, xml, webpages, Google, GitHub, video/audio, images and other data types, or use conversation history as additional context provided to the model.
• Built-in vector databases support and automated files and data embedding.
• Image generation via models like DALL-E, gpt-image, Imagen, Gemini, and Nano Banana.
• Video generation via models like Veo3 and Sora2.
• Internet access via DuckDuckGo, Google and Microsoft Bing.
• Speech synthesis via Microsoft Azure, Google, Eleven Labs and OpenAI Text-To-Speech services.
• Speech recognition via OpenAI Whisper, Google and Microsoft Speech Recognition.
• Plugins support with built-in plugins like Files I/O, Code Interpreter, Web Search, Google, Facebook, X/Twitter, Slack, Telegram, GitHub, MCP, and many more.
• MCP support.
• Real-time video camera capture in Vision mode.
• Image analysis via vision models.
• Included support features for individuals with disabilities: customizable keyboard shortcuts, voice control, and translation of on-screen actions into audio via speech synthesis.
• Handles and stores the full context of conversations (short and long-term memory).
• Integrated calendar, day notes and search in contexts by selected date.
• Tools and commands execution (via plugins: access to the local filesystem, Python Code Interpreter, system commands execution, and more).
• Custom commands creation and execution.
• Crontab / Task scheduler included.
• Built-in real-time Python Code Interepreter.
• Manages files and attachments with options to upload, download, and organize.
• Context history with the capability to revert to previous contexts (long-term memory).
• Allows you to easily manage prompts with handy editable presets.
• Provides an intuitive operation and interface.
• Includes a notepad.
• Includes simple painter / drawing tool.
• Includes an node-based Agents Builder.
• Supports multiple languages.
• Requires no previous knowledge of using AI models.
• Fully configurable.
• Themes support.
• Real-time code syntax highlighting.
• Built-in token usage calculation.
• Possesses the potential to support future OpenAI models.
• Open source; source code is available on GitHub.
• Utilizes the user's own API key.
• and many more.

The application is free, open-source, and runs on PCs with Linux, Windows 10, Windows 11 and Mac. Full Python source code is available on GitHub.

PyGPT uses the user's API key - to use the GPT models, you must have a registered OpenAI account and your own API key. Local models do not require any API keys.

You can also use built-it LlamaIndex support to connect to other Large Language Models (LLMs), such as those on HuggingFace. Additional API keys may be required.

Installation

Binaries (Linux, Windows 10 and 11)

You can download compiled binary versions for Linux and Windows (10/11).

PyGPT binaries require a PC with Windows 10, 11, or Linux. Simply download the installer or the archive with the appropriate version from the download page at https://pygpt.net, extract it, or install it, and then run the application. A binary version for Mac is not available, so you must run PyGPT from PyPi or from the source code on Mac. Currently, only 64-bit binaries are available.

Linux version requires GLIBC >= 2.35.

Microsoft Store (Windows)

For Windows 10/11, you can install PyGPT directly from Microsoft Store:

Link to MS Store: https://apps.microsoft.com/detail/XP99R4MX3X65VQ

AppImage (Linux)

You can download the latest PyGPT AppImage for Linux from the release page:

Releases: https://github.com/szczyglis-dev/py-gpt/releases

Tip: Remember to give execution permissions to the downloaded file:

chmod +x ./PyGPT-X.X.X-x86_64.AppImage

To manage future updates you can use AppImageUpdate tool:

You can download it from: https://github.com/AppImage/AppImageUpdate/releases

After downloading, run the following command in terminal:

appimageupdatetool ./PyGPT-X.X.X-x86_64.AppImage

Snap Store (Linux)

You can install PyGPT directly from Snap Store:

sudo snap install pygpt

To manage future updates use:

sudo snap refresh pygpt

Using camera: to use camera in Snap version you must connect the camera with:

sudo snap connect pygpt:camera

Using microphone: to use microphone in Snap version you must connect the microphone with:

sudo snap connect pygpt:audio-record :audio-record
sudo snap connect pygpt:alsa

Using audio output: to use audio output in Snap version you must connect the audio with:

sudo snap connect pygpt:audio-playback
sudo snap connect pygpt:alsa

Connecting IPython in Docker in Snap version:

To use IPython in the Snap version, you must connect PyGPT to the Docker daemon:

sudo snap connect pygpt:docker-executables docker:docker-executables

sudo snap connect pygpt:docker docker:docker-daemon

PyPi (pip)

The application can also be installed from PyPi using pip install:

1. Create virtual environment:

python3 -m venv venv
source venv/bin/activate

2. Install from PyPi:

pip install pygpt-net

3. Once installed run the command to start the application:

pygpt

Running from GitHub source code

An alternative method is to download the source code from GitHub and execute the application using the Python interpreter (>=3.10, <3.14).

Install with pip

1. Clone git repository or download .zip file:

git clone https://github.com/szczyglis-dev/py-gpt.git
cd py-gpt

2. Create a new virtual environment:

python3 -m venv venv
source venv/bin/activate

3. Install requirements:

pip install -r requirements.txt

4. Run the application:

python3 run.py

Install with Poetry

1. Clone git repository or download .zip file:

git clone https://github.com/szczyglis-dev/py-gpt.git
cd py-gpt

2. Install Poetry (if not installed):

pip install poetry

3. Create a new virtual environment that uses Python 3.10:

poetry env use python3.10
poetry shell

or (Poetry >= 2.0):

poetry env use python3.10
poetry env activate

4. Install requirements:

poetry install

5. Run the application:

poetry run python3 run.py

Tip: you can use PyInstaller to create a compiled version of the application for your system (required version >= 6.0.0).

Troubleshooting

If you have a problems with xcb plugin with newer versions of PySide on Linux, e.g. like this:

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. 
Reinstalling the application may fix this problem.

...then install libxcb:

sudo apt install libxcb-cursor0

If you have a problems with audio on Linux, then try to install portaudio19-dev and/or libasound2:

sudo apt install portaudio19-dev

sudo apt install libasound2
sudo apt install libasound2-data 
sudo apt install libasound2-plugins

Problems with GLIBC on Linux

If you encounter error:

Error loading Python lib libpython3.10.so.1.0: dlopen: /lib/x86_64-linux-gnu/libm.so.6: version GLIBC_2.35 not found (required by libpython3.10.so.1.0)

when trying to run the compiled version for Linux, try updating GLIBC to version 2.35, or use a newer operating system that has at least version 2.35 of GLIBC.

Access to camera in Snap version:

sudo snap connect pygpt:camera

Access to microphone in Snap version:

To use microphone in Snap version you must connect the microphone with:

sudo snap connect pygpt:audio-record :audio-record

Snap and AppArmor permission denied

Snap installs AppArmor profiles for each application by default. The profile for PyGPT is created at:

/var/lib/snapd/apparmor/profiles/snap.pygpt.pygpt

The application should work with the default profile; however, if you encounter errors like:

PermissionError: [Errno 13] Permission denied: '/etc/httpd/conf/mime.types'

add the appropriate access rules to the profile file, for example:

# /var/lib/snapd/apparmor/profiles/snap.pygpt.pygpt

...

/etc/httpd/conf/mime.types r

and reload the profiles.

Alternatively, you can try removing snap and reinstalling it:

sudo snap remove --purge pygpt

sudo snap install pygpt

Access to a microphone and audio in Windows version:

If you have a problems with audio or a microphone in the non-binary PIP/Python version on Windows, check to see if FFmpeg is installed. If it's not, install it and add it to the PATH. You can find a tutorial on how to do this here: https://phoenixnap.com/kb/ffmpeg-windows. The binary version already includes FFmpeg.

Windows and VC++ Redistributable

On Windows, the proper functioning requires the installation of the VC++ Redistributable, which can be found on the Microsoft website:

https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist

The libraries from this environment are used by PySide6 - one of the base packages used by PyGPT. The absence of the installed libraries may cause display errors or completely prevent the application from running.

It may also be necessary to add the path C:\path\to\venv\Lib\python3.x\site-packages\PySide6 to the PATH variable.

WebEngine/Chromium renderer and OpenGL problems

If you have a problems with WebEngine / Chromium renderer you can force the legacy mode by launching the app with command line arguments:

python3 run.py --legacy=1

and to force disable OpenGL hardware acceleration:

python3 run.py --disable-gpu=1

You can also manualy enable legacy mode by editing config file - open the %WORKDIR%/config.json config file in editor and set the following options:

"render.engine": "legacy",
"render.open_gl": false,

Other requirements

For operation, an internet connection is needed (for API connectivity), a registered OpenAI account, and an active API key that must be input into the program. Local models, such as Llama3 do not require OpenAI account and any API keys.

Debugging and logging

Please go to Debugging and Logging section for instructions on how to log and diagnose issues in a more detailed manner.

Quick Start

Setting-up API Key(s)

You can configure API keys for various providers, such as OpenAI, Anthropic, Google, xAI, Perplexity, OpenRouter, and more. This flexibility allows you to use different providers based on your needs.

During the initial setup, configure your API keys within the application.

To do so, navigate to the menu:

Config -> Settings -> API Keys

Here, you can add or manage API keys for any supported provider.

Configuring Provider

1. Select the Provider: Choose a tab with provider.
2. Enter the API Key: Paste the corresponding API key for the selected provider.

Example

• OpenAI: Obtain your API key by registering on the OpenAI website: https://platform.openai.com and navigating to https://platform.openai.com/account/api-keys.
• Anthropic, Google, etc.: Follow similar steps on their respective platforms.

Note: The ability to use models or services depends on your access level with the respective provider. If you wish to use custom API endpoints or local APIs that do not require API keys, simply enter any value into the API key field to bypass prompts about an empty key.

Work modes

Chat

+ Inline Vision and Image generation

In PyGPT, this mode mirrors ChatGPT, allowing you to chat with models like GPT-5, GPT-4, o1, o3, Claude, Gemini, Grok, Perplexity (Sonar), Deepseek, and more. It works with the OpenAI SDK using the Responses API and ChatCompletions API. You can also use SDKs from Google GenAI, Anthropic, or xAI if the native SDK is enabled. You can set the endpoint for ChatCompletions in Config -> Settings -> API Keys`.

Tip: This mode uses the provider SDK directly. If there's no native client built into the app, models like Sonar, or Llama3 are supported in Chat mode via LlamaIndex or OpenAI-compatible API endpoints. The app automatically switches to these endpoints when using non-OpenAI models. You can enable or disable the use of the native API SDK (per provider) in Settings -> API Keys. If the native SDK is disabled, the OpenAI SDK will be used via the compatible ChatCompletions API endpoint.

Currently built-in native clients:

• Anthropic SDK
• OpenAI SDK
• Google GenAI SDK
• xAI SDK

The main part of the interface is a chat window where you see your conversations. Below it is a message box for typing. On the right side, you can set up or change the model and system prompt. You can also save these settings as presets to easily switch between models or tasks.

Above where you type your messages, the interface shows you the number of tokens your message will use up as you type it – this helps to keep track of usage. There is also a feature to attach and upload files in this area. Go to the Files and Attachments section for more information on how to use attachments.

Vision: If you want to send photos from your disk or images from your camera for analysis, and the selected model does not support Vision, you must enable the Vision (inline) plugin in the Plugins menu. This plugin allows you to send photos or images from your camera for analysis in any Chat mode.

With this plugin, you can capture an image with your camera or attach an image and send it for analysis to discuss the photograph:

Image generation: If you want to generate images directly in chat you must enable plugin Image generation (inline) in the Plugins menu. Plugin allows you to generate images in Chat mode:

Chat with Files (LlamaIndex)

This mode enables chat interaction with your documents and entire context history through conversation. It seamlessly incorporates LlamaIndex into the chat interface, allowing for immediate querying of your indexed documents.

Tip: If you do not want to call tools/commands, disable the checkbox +Tools. It will speed up the response time when using local models. You can also enable the ReAct agent for tool calls in: Settings -> Indexes / LlamaIndex -> Chat -> Use ReAct agent for Tool calls in Chat with Files mode. Stream mode is disabled if the ReAct agent and +Tools checkbox are active.

Querying single files

You can also query individual files "on the fly" using the query_file command from the Files I/O plugin. This allows you to query any file by simply asking a question about that file. A temporary index will be created in memory for the file being queried, and an answer will be returned from it. From version 2.1.9 similar command is available for querying web and external content: Directly query web content with LlamaIndex.

For example:

If you have a file: data/my_cars.txt with content My car is red.

You can ask for: Query the file my_cars.txt about what color my car is.

And you will receive the response: Red.

Note: this command indexes the file only for the current query and does not persist it in the database. To store queried files also in the standard index you must enable the option Auto-index readed files in plugin settings. Remember to enable + Tools checkbox to allow usage of tools and commands from plugins.

Using Chat with Files mode

In this mode, you are querying the whole index, stored in a vector store database. To start, you need to index (embed) the files you want to use as additional context. Embedding transforms your text data into vectors. If you're unfamiliar with embeddings and how they work, check out this article:

https://stackoverflow.blog/2023/11/09/an-intuitive-introduction-to-text-embeddings/

For a visualization from OpenAI's page, see this picture:

Source: https://cdn.openai.com/new-and-improved-embedding-model/draft-20221214a/vectors-3.svg

To index your files, simply copy or upload them into the data directory and initiate indexing (embedding) by clicking the Index all button, or right-click on a file and select Index.... Additionally, you have the option to utilize data from indexed files in any Chat mode by activating the Chat with Files (LlamaIndex, inline) plugin.

After the file(s) are indexed (embedded in vector store), you can use context from them in chat mode:

Built-in file loaders:

Files:

• CSV files (csv)
• Epub files (epub)
• Excel .xlsx spreadsheets (xlsx)
• HTML files (html, htm)
• IPYNB Notebook files (ipynb)
• Image (vision) (jpg, jpeg, png, gif, bmp, tiff, webp)
• JSON files (json)
• Markdown files (md)
• PDF documents (pdf)
• Plain-text files (txt)
• Video/audio (mp4, avi, mov, mkv, webm, mp3, mpeg, mpga, m4a, wav)
• Word .docx documents (docx)
• XML files (xml)

Web/external content:

• Bitbucket
• ChatGPT Retrieval Plugin
• GitHub Issues
• GitHub Repository
• Google Calendar
• Google Docs
• Google Drive
• Google Gmail
• Google Keep
• Google Sheets
• Microsoft OneDrive
• RSS
• SQL Database
• Sitemap (XML)
• Twitter/X posts
• Webpages (crawling any webpage content)
• YouTube (transcriptions)

You can configure data loaders in Settings / Indexes / LlamaIndex / Data Loaders by providing list of keyword arguments for specified loaders. You can also develop and provide your own custom loader and register it within the application.

LlamaIndex is also integrated with context database - you can use data from database (your context history) as additional context in discussion. Options for indexing existing context history or enabling real-time indexing new ones (from database) are available in Settings / Indexes / LlamaIndex section.

WARNING: remember that when indexing content, API calls to the embedding model are used. Each indexing consumes additional tokens. Always control the number of tokens used on the provider's page.

Tip: Using the Chat with Files mode, you have default access to files manually indexed from the /data directory. However, you can use additional context by attaching a file - such additional context from the attachment does not land in the main index, but only in a temporary one, available only for the given conversation.

Token limit: When you use Chat with Files in non-query mode, LlamaIndex adds extra context to the system prompt. If you use a plugins (which also adds more instructions to system prompt), you might go over the maximum number of tokens allowed. If you get a warning that says you've used too many tokens, turn off plugins you're not using or turn off the "+ Tools" option to reduce the number of tokens used by the system prompt.

Available vector stores (provided by LlamaIndex):

- ChromaVectorStore
- ElasticsearchStore
- PinecodeVectorStore
- QdrantVectorStore
- RedisVectorStore
- SimpleVectorStore

You can configure selected vector store by providing config options like api_key, etc. in Settings -> LlamaIndex window. See the section: Configuration / Vector stores for configuration reference.

Configuring data loaders

In the Settings -> LlamaIndex -> Data loaders section you can define the additional keyword arguments to pass into data loader instance. See the section: Configuration / Data Loaders for configuration reference.

Chat with Audio

This mode works like the Chat mode but with native support for audio input and output using a Realtime and Live APIs. In this mode, audio input and output are directed to and from the model directly, without the use of external plugins. This enables faster and better audio communication.

Currently, in beta.

At this moment, only OpenAI real-time models (via the Realtime API) and Google Gemini real-time models (via the Live API) are supported.

Research

This mode (when using Sonar and R1 models) operates using the Perplexity API: https://perplexity.ai.

It allows for deep web searching and utilizes Sonar models, available in Perplexity AI.

It requires a Perplexity API key, which can be generated at: https://perplexity.ai.

From version 2.5.27 also OpenAI deep-research models are available in this mode.

Completion

An older mode of operation that allows working in the standard text completion mode. However, it allows for a bit more flexibility with the text by enabling you to initiate the entire discussion in any way you like.

Similar to chat mode, on the right-hand side of the interface, there are convenient presets. These allow you to fine-tune instructions and swiftly transition between varied configurations and pre-made prompt templates.

Additionally, this mode offers options for labeling the AI and the user, making it possible to simulate dialogues between specific characters - for example, you could create a conversation between Batman and the Joker, as predefined in the prompt. This feature presents a range of creative possibilities for setting up different conversational scenarios in an engaging and exploratory manner.

From version 2.0.107 the davinci models are deprecated and has been replaced with gpt-3.5-turbo-instruct model in Completion mode.

Image and video generation

PyGPT enables quick and easy image creation with models like DALL-E 3, gpt-image-1, Imagen 3/4, and Nano Banana, as well as video generation using Veo3 and Sora2. Generating images and videos is akin to a chat conversation - a user's prompt triggers the generation, followed by downloading, saving to the computer, and displaying the image onscreen. You can send raw prompt to the model in Image generation mode or ask the model for the best prompt.

Image generation using image models is also available in every mode via plugin Image Generation (inline). Just ask any model, in any mode, like e.g. GPT or Gemini to generate an image and it will do it inline, without need to mode change.

If you want to generate images directly in chat you must enable plugin Image generation (inline) in the Plugins menu. Plugin allows you to generate images in Chat mode:

For OpenAI models, you can also enable remote image generation in Config -> Settings -> Remote Tools. If enabled, image generation will be available natively within the conversation, without plugins, in Chat mode.

To use Imagen models you must enable VertexAI in Config -> Settings -> API Keys -> Google -> Advanced options.

Remix, Edit, or Extend

To remix or extend from a previous image or video instead of creating a new one from scratch, enable the Remix/Extend option checkbox in the toolbox. The last generated image or video in the current context will be used as a reference for your prompt, allowing you to request changes to the generated content. If the Remix/Extend option is enabled, uploading an image attachment as a reference will not take effect.

Raw mode

There is an option for switching prompt generation mode.

If Raw Mode is enabled, a model will receive the prompt exactly as you have provided it. If Raw Mode is disabled, a model will generate the best prompt for you based on your instructions.

Image storage

Once you've generated an image, you can easily save it anywhere on your disk by right-clicking on it. You also have the options to delete it or view it in full size in your web browser.

Tip: Use presets to save your prepared prompts. This lets you quickly use them again for generating new images later on.

The app keeps a history of all your prompts, allowing you to revisit any session and reuse previous prompts for creating new images.

Images are stored in img directory in PyGPT user data folder.

Assistants

This mode uses the OpenAI's Assistants API.

This mode expands on the basic chat functionality by including additional external tools like a Code Interpreter for executing code, Retrieval Files for accessing files, and custom Functions for enhanced interaction and integration with other APIs or services. In this mode, you can easily upload and download files. PyGPT streamlines file management, enabling you to quickly upload documents and manage files created by the model.

Setting up new assistants is simple - a single click is all it takes, and they instantly sync with the OpenAI API. Importing assistants you've previously created with OpenAI into PyGPT is also a seamless process.

In Assistant mode you are allowed to storage your files in remote vector store (per Assistant) and manage them easily from app:

Please note that token usage calculation is unavailable in this mode. Nonetheless, file (attachment) uploads are supported. Simply navigate to the Files tab to effortlessly manage files and attachments which can be sent to the OpenAI API.

Vector stores (via Assistants API)

Assistant mode supports the use of external vector databases offered by the OpenAI API. This feature allows you to store your files in a database and then search them using the Assistant's API. Each assistant can be linked to one vector database—if a database is linked, all files uploaded in this mode will be stored in the linked vector database. If an assistant does not have a linked vector database, a temporary database is automatically created during the file upload, which is accessible only in the current thread. Files from temporary databases are automatically deleted after 7 days.

To enable the use of vector stores, enable the Chat with Files checkbox in the Assistant settings. This enables the File search tool in Assistants API.

To manage external vector databases, click the DB icon next to the vector database selection list in the Assistant creation and editing window (screen below). In this management window, you can create a new vector database, edit an existing one, or import a list of all existing databases from the OpenAI server:

You can define, using Expire days, how long files should be automatically kept in the database before deletion (as storing files on OpenAI incurs costs). If the value is set to 0, files will not be automatically deleted.

The vector database in use will be displayed in the list of uploaded files, on the field to the right—if a file is stored in a database, the name of the database will be displayed there; if not, information will be shown indicating that the file is only accessible within the thread:

Agent (LlamaIndex)

Mode that allows the use of agents offered by LlamaIndex.

Includes built-in agents (Workflow):

• FunctionAgent
• ReAct
• Structured Planner (sub-tasks)
• CodeAct (connected to Code Interpreter plugin)
• Supervisor + worker

Includes built-in agents (Legacy):

• OpenAI Assistants

In the future, the list of built-in agents will be expanded.

You can create your own types (workflows/patterns) using the built-in visual node-based editor found in the Tools -> Agents Builder.

You can also create your own agent by creating a new provider that inherits from pygpt_net.provider.agents.base.

Tools and Plugins

In this mode, all commands from active plugins are available (commands from plugins are automatically converted into tools for the agent on-the-fly).

RAG - using indexes

If an index is selected in the agent preset, a tool for reading data from the index is automatically added to the agent, creating a RAG automatically.

Multimodality is currently unavailable, only text is supported. Vision support will be added in the future.

Loop / Evaluate Mode

You can run the agent in autonomous mode, in a loop, and with evaluation of the current output. When you enable the Loop / Evaluate checkbox, after the final response is given, the quality of the answer will be rated on a percentage scale of 0% to 100% by another agent. If the response receives a score lower than the one expected (set using a slider at the bottom right corner of the screen, with a default value 75%), a prompt will be sent to the agent requesting improvements and enhancements to the response.

Setting the expected (required) score to 0% means that the response will be evaluated every time the agent produces a result, and it will always be prompted to self-improve its answer. This way, you can put the agent in an autonomous loop, where it will continue to operate until it succeeds.

You can choose between two methods of evaluation:

• By the percentage of tasks completed
• By the accuracy (score) of the final response

You can set the limit of steps in such a loop by going to Settings -> Agents and experts -> LlamaIndex agents -> Max evaluation steps . The default value is 3, meaning the agent will only make three attempts to improve or correct its answer. If you set the limit to zero, there will be no limit, and the agent can operate in this mode indefinitely (watch out for tokens!).

You can change the prompts used for evaluating the response in Settings -> Prompts -> Agent: evaluation prompt in loop. Here, you can adjust it to suit your needs, for example, by defining more or less critical feedback for the responses received.

Agent (OpenAI)

The mode operates on the openai-agents library integrated into the application:

https://github.com/openai/openai-agents-python

It allows running agents for OpenAI models and models compatible with the OpenAI API.

In this mode, you can use pre-configured Experts in Expert mode presets - they will be launched as agents (in the openai_agents_experts type, which allows launching one main agent and subordinate agents to which queries will be appropriately directed).

Agent types (workflows/patterns):

• Agent with experts - uses attached experts as sub-agents
• Agent with experts + feedback - uses attached experts as sub-agents + feedback agent in a loop
• Agent with feedback - single agent + feedback agent in a loop
• Planner - planner agent, 3 sub-agents inside: planner, base agent + feedback
• Research bot - researcher, 3 sub-agents inside: planner, searcher and writer as base agent
• Simple agent - a single agent.
• Evolve - in each generation (cycle), the best response from a given parent agent is selected; in the next generation, the cycle repeats.
• B2B - bot-to-bot communication, involving two bots interacting with each other while keeping a human in the loop.
• Supervisor + Worker - one agent (supervisor) acts as a bridge between the user and the second agent (worker). The user provides a query to the supervisor, who then sends instructions to the worker until the task is completed by the worker.

You can create your own types (workflows/patterns) using the built-in visual node-based editor found in the Tools -> Agents Builder.

There are also predefined presets added as examples:

• Coder
• Experts agent
• Planner
• Researcher
• Simple agent
• Writer with Feedback
• 2 bots
• Supervisor + worker

In the Agents (OpenAI) mode, all remote tools are available for the base agent according to the configuration in the Config -> Settings -> Remote tools menu.

Remote tools for experts can be selected separately for each expert in the preset configuration.

Local tools (from plugins) are available for agents and experts according to the enabled plugins, as in other modes.

In agents with feedback and plans, tools can be allowed in a preset configuration for each agent. They also have separate prompts that can be configured in presets.

Description of how different types of agents work:

Below is a pattern for how different types of agents work. You can use these patterns to create agents for different tasks by modifying the appropriate prompts in the preset for the specific task.

Simple Agent

• The agent completes its task and then stops working.

Agent with Feedback

• The first agent answers a question.
• The second agent (feedback) evaluates the answer and, if necessary, goes back to the first agent to enforce corrections.
• The cycle repeats until the feedback agent is satisfied with the evaluation.

Agent with Experts

• The agent completes the assigned task on its own or delegates it to the most suitable expert (another agent).

Agent with Experts + Feedback

• The first agent answers a question or delegates it to the most suitable expert.
• The second agent (feedback) evaluates and, if necessary, goes back to the first agent to enforce corrections.
• The cycle repeats until the feedback agent is satisfied with the evaluation.

Research Bot

• The first agent (planner) prepares a list of phrases to search.
• The second agent (search) finds information based on the phrases and creates a summary.
• The third agent (writer) prepares a report based on the summary.

Planner

• The first agent (planner) breaks down a task into sub-tasks and sends the list to the second agent.
• The second agent performs the task based on the prepared task list.
• The third agent, responsible for feedback, evaluates, requests corrections if needed, and sends the request back to the first agent. The cycle repeats.

Evolve

• You select the number of agents (parents) to operate in each generation (iteration).
• Each agent prepares a separate answer to a question.
• The best agent (producing the best answer) in a generation is selected by the next agent (chooser).
• Another agent (feedback) verifies the best answer and suggests improvements.
• A request for improving the best answer is sent to a new pair of agents (new parents).
• From this new pair, the best answer is selected again in the next generation, and the cycle repeats.

B2B

• A human provides a topic for discussion.
• Bot 1 generates a response and sends it to Bot 2.
• Bot 2 receives the response from Bot 1 as input, provides an answer, and sends t