AI Pals Engine News

[p]Happy Sunday! You might recall we used an MCP Tool to generate images in the last Devlog about Behaviors. This Devlog takes a closer look at how that actually works.[/p][p]Tools/Agents/Skills are a hot topic right now. Frontier models have improved drastically in recent months, particularly regarding coding tasks. But most of these advancements can also be used in a more general way to do all kinds of things. We're getting dangerously close to the point where I can replace myself with a coding agent and simply supervise it! Anyway, we will see this expand into more capable open models in the future as more training data gets integrated for this task, in addition to native capabilities within the chat format.[/p][p]A tool call to retrieve realtime information from the web[/p][p][/p][h2]🛠️What is Tool Calling?[/h2][p]Ultimately, it is just a program following a protocol (AI Pals Engine uses MCP - Model Context Protocol): the Text Model generates the command and the tool executes it.[/p][p]Imagine a tool that lets the AI execute Windows commands directly. Is it smart to give your AI that kind of power? Probably not.[/p][p][/p][h2]✅Can I do Tool Calling in AI Pals Engine?[/h2][p]Yes, support is much broader after the recent update and hopefully you can add any tool. But as mentioned within the app, always use only trusted tools. They carry a lot of risk![/p][p]You can opt for a Single-Step approach (one tool call) or a Multi-Step workflow where the AI executes a chain of calls to complete a task.[/p][p]You also have full control over how the data is handled: whether the result is added to the context, used immediately in a response, or both. There are also different viewing modes: Raw (to see every detail) or Compact (to save tokens and keep the format pretty).[/p][p]Beyond the basics, you can add tools into group chats to create teams of "Agent-like" characters. There are also special tools like send_message_to_user (for status reports) and end_task (to stop chains). To make sure everything runs smoothly, AI Pals Engine uses grammar enforcement by default, guiding the model to ensure it always produces valid commands.[/p][h3]Example 1 – Realtime Info & Grounding [/h3][p]Large Language Models are famously "frozen in time", they only know what they learned during training. If you ask them about news from yesterday, they might hallucinate or give you outdated info.[/p][p]This is where Grounding comes in. Grounding essentially means anchoring the AI's response in reality by providing it with external, up-to-date evidence. You can do this via the Memory Feature (providing text documents), by using a real-time search tool like the example above (the example utilizes SearXNG) or by manually providing the info. Instead of using the send_message_to_user tool, we changed the mode of the search tool to "ActOnResult" to compile a summarized answer using the retrieved information.[/p][h3]Example 2 – Let your AI do some work by chaining tool calls[/h3][p]Why browse the web yourself when your AI Pal can do it for you?[/p][p][/p][p]By utilizing Playwright (a general browser automation tool), the AI can navigate the web, click buttons and interact with pages. The magic here is the Chaining capability. The AI doesn't just fire one command and quit, it intelligently strings together a sequence of actions (browsing, clicking and reading) to complete complex tasks entirely on its own. But don't get too excited, a small model will fail to do things (if it wasn't trained especially for it) and the context usage can be really high for this task (a lot of computation).[/p][h3]Other Examples[/h3][p]Like with Behaviors, Tool Calling allows for endless possibilities to do more than just text generation with your Char. As mentioned in the intro, these capabilities will only improve in the near future.[/p][p]I personally use it with Github Tools and to enter or modify Notes/TODOs. I plan to experiment much further - for example, generating predefined choices that get automatically forwarded to a widget panel. We could even implement a standard for interpreting returned results to trigger other panels or display info. This would better support both game-like and general interactive functionality, such as tracking game states, inventory or skill stats, system states, user data and other dynamic information displays.[/p][p]Dialogue Options[/p][p][/p][h2]🌱Future Plans[/h2][p]This thread is about how to make Tools/Skills discoverable. For Steam, the most convenient way would probably be the Workshop, so that we can just pick the Skills we want in the Char Config. But the main issue is: how to make it secure? We want the tools to be powerful, but at the same time, they can be used to harm your system in many different ways.[/p][p]For now, the best approach is likely to make the tools of the AI Char shareable to the Workshop with metadata on which tools were used and where to find them.[/p][p]Another idea is to add a UI panel to temporarily disable/enable tools (e.g. the web search tool). Sometimes you need it, sometimes you don't and it is probably faster to use a panel than to switch the AI Char entirely (of course you can leave that up to the AI, but I usually prefer to have the control over it).[/p][p]Let’s keep an eye on advances and reassess in a few months, once more models are available on newer standards.[/p][h2]🌅What's next for AI Pals Engine?[/h2][p]Hopefully, this Devlog gave you a better idea about the Tool concept in AI Pals Engine. You can also use tools manually in your Behavior workflows. For the Text AI, there is just a basic tool call example available within the app (chat-general-toolcall), but with some tweaks and by adding tools, you can make a really powerful Widget![/p][p]The question remains: Would a skill workshop be a good addition? Also, what about security? You can comment here or use the mentioned thread.

Here are some points for the next updates:[/p]

• [p]TTS SPEED Config[/p]
• [p]STT Wake Word[/p]
• [p]Basic Multi-Gpu config[/p]
• [p]NoSteam Option (disable Steam Time Tracking) and Autostart[/p]
• [p]Better Audio and Memory+Dynamic Tokens integrations[/p]
• [p]More of the following: Help pages, Internationalization, Fixes, QoL Improvements[/p]

[h2]📝Changelog (summarized from last to V0.8.11.0):[/h2]

• [p]TTS: Improved Piper TTS support (load any model/language).[/p]
• [p]TTS: Fixed Kokoro not working without eSpeak.[/p]
• [p]TTS: Fixed default Kokoro not using the correct text-split strategy.[/p]
• [p]TTS: Optimizations and better handling of sentences including the "-" character.[/p]
• [p]Widgets: Added option to automatically load the last conversation on startup.[/p]
• [p]MCP: Reworked "Add MCP Server" to improve usability.[/p]
• [p]MCP: Added support for Environment Variables (needed for custom Python env servers).[/p]
• [p]MCP: Added basic HTTP support.[/p]
• [p]MCP: Added support to dynamically generate the method for the McpAction Behavior Node.[/p]
• [p]Tools: Added OAI Image tool.[/p]
• [p]Tools: Added Web URL POST tool.[/p]
• [p]Tools: Reworked Character Tool Config to improve usability.[/p]
• [p]Tools: Fixed send_message_to_user not outputting correctly in Raw Tool Call Mode.[/p]
• [p]Workshop: Character Tool allow-lists are now exported (Tool Call Mode defaults to "None" for security).[/p]
• [p]Workshop: Fixed Workshop items not being deletable if removed manually from Steam.[/p]
• [p]General: Added more help text and tooltips.[/p]
• [p]General: Added warning when exiting the app if a download is active.[/p]
• [p]Performance: Fixed performance issues when loading many entries in the Memory View.[/p]

[p]Happy Sunday! Behaviors are one of the things I'm most excited about. They form the backbone of a truly dynamic Pal - enabling automation, unlocking deep modding possibilities, serving as a strong starting point for custom functionality and further enrich our Widgets. That's why I'd like to dive into them a bit more deeply, with real examples. At the bottom, you'll also find the changelog, detailing the fixes from the recent smaller updates.[/p][p][/p]

[p]Behaviors are essential to the core, every update on them opens new doors for Pals and Widgets. The real magic shines when you take them further: crafting custom behavior, testing limits and adding/creating entirely new things.[/p]

[p][/p][p]Example - A Widget Behavior that initiates the generation of an image (external tool) to visualize the current scene:[/p][p][/p][h2]✨ Moving beyond Language Models in AI Pals Engine[/h2][p]Language models are fantastic: they handle language processing, can mimic personality, keep conversations flowing and understand high‑level intent. But we need[/p][p][/p][p]While language models act as the soul or brain of a Pal, we also need a "nervous system" that executes actions like animations, movements, sounds and more. Not only does this system power the Pal, but it also enriches our Widgets with more interactivity and functionality.[/p][p]Combining language models with Behaviors gets us closer to a true mind‑and‑body avatar. Behaviors serve as the nervous system: fast, predictable and rule‑based. They’re built on Behavior Trees, a widely used AI decision-making method in gaming (More AI on top of AI in AI Pals).[/p][p][/p][h2]🧩 Are they available in AI Pals Engine?[/h2][p]Yes, you can edit them, create them and share them. You'll find them by clicking on Behaviors in your Pal's Avatar Config within the Widget Config (an avatar has to be set). You can even reuse your existing MCP tools as mechanisms to trigger actions on other systems.[/p][p]And yes, Behavior Trees aren’t exactly intuitive at first, especially given the current basic implementation and editor. But they're a well-established foundation. We can later add an ability-like system where we simply pick pre-built trees. And AI can help build them or even better: what if the AI modifies its own “nervous system”, dynamically generating or evolving Behavior Trees on the fly?[/p][p]Let’s cook some more on it. Anyway, enough theory let’s get into some examples.[/p][h3]Pal Example[/h3][p]This is my favorite Pal (Example Widget Name: No AI 2 – Cat), it makes me happy, you can pet it and it plays a sound via Behaviors 🐱:[/p][p][/p][p]What if we want it to meow randomly? Easy we just replace the Interaction-Trigger Node with a Random-Delay Node:[/p][p] [/p][p]I use it personally as a notification system. I get 10 meows for every review and 1 meow for a new entry in the comm hub. How?[/p][p]A custom-coded MCP Tool that checks both counts - combined with this Behavior (+ a Repeat Node set to 10 repeats to trigger 10 meows).[/p][p][/p][h3]Widget Example[/h3][p][/p][p]As mentioned Behaviors can also serve to improve the Widgets. Let's take a look at the Advanced 5 – Dice Adventure example. Here, we use Behaviors to inject dice rolls and introduce real randomness into the AI model’s decision-making. For instance, if we encounter a monster and roll a negative outcome, your character could die right then and there (Make sure not to use a completely positive biased model).[/p][p]In the future, I plan to do more experiments around this and more complex game rules. Maybe introducing state systems to manage story arcs, inventory and abilities, all of which could influence the randomization and deepen the narrative experience.[/p][p][/p][h3]Image Example[/h3][p]So, how can we recreate the example from the beginning of the post? It follows the exact same logic and is pretty easy, the overall steps are:[/p]

1. [p]Determine when we want to get the image. After the AI completes a message or do you prefer triggering it manually via an action button? (Use the [c]GetLastEventDataAction[/c] node)[/p]
2. [p]Retrieve the image via a custom MCP Tool (which also enables Pals to generate images) or through an endpoint using an [c]McpAction[/c] node ([c]AIP-MCP.get_from_url_image)[/c][/p]
3. [p]Set the new image as the avatar of a decoration Pal or your current Pal (use the [c]SetPalAvatarAction[/c] node).[/p]

[p]The tricky part is integrating a local running image gen functionality. The plan is to add a tutorial to the Behavior help section soon - making it super-easy to recreate this on your own hardware, without the usual hassles and dependency headaches (or in Todd's words: it just works) + the importable Behavior.[/p][p][/p][h2]🌅What's next for AI Pals Engine?[/h2][p]I hope I was able to communicate the power of these concepts. We'll definitely need some quality-of-life and usability improvements. If you want to experiment, go ahead, particularly let me know if any crucial nodes feel missing![/p][p]Release means a lot of troubleshooting that costs a lot of time and are hard to predict. Sometimes AI Pals Engine fails, but sometimes even Windows fails with corrupt .NET environments. So slowly we can start on new things again. Here are some points for the next time, shifting to the things we are discussing in the Comm Hub/Discord, like the following:[/p]

• [p]Better MCP Config and more Char synergy + Workshop support[/p]
• [p]Multilingual TTS[/p]
• [p]TTS SPEEEED Config[/p]
• [p]STT Wake Word[/p]
• [p]Basic Multi-Gpu config[/p]
• [p]NoSteam Option (disable Steam Time Tracking)[/p]
• [p]Autostart AI Pals Engine and an option to pick up the last conversation automatically in the widget[/p]
• [p]Better Audio and Memory integrations[/p]
• [p]More of the following: Help pages, Internationalization, Fixes, QoL Improvements[/p]

[p]Let me know what you need and if you’ve had the chance to experiment with Behaviors yet![/p][p][/p][h3]Changelog V0.8.7.9[/h3][p]Summarized from smaller updates deployed over the week:[/p]

• [p]appLog.txt is now more verbose in case of unhandled exceptions and a proper error message got added[/p]
• [p]Debug Window: It is now less verbose by default (You can reenable it in Settings -> Chat)[/p]
• [p]Many UI Naming and Usability improvements like:[/p][p]Avatar renamed to Pal Avatar in Widget Config, Added Copy Text Button in Debug Window, Proper Warnings for Widget Foreground and Mouse Interaction Settings, Chars -> Added Details to context menu, Preset -> better distinction of set vs unset model and added button to reset the default preset, better clipboard error handling, additional messages to explain the app's functionality[/p]
• [p]FIX STT: Button not changing the color properly when in Record mode (main app)[/p]
• [p]FIX Char Details: A case in which Pending Changes were automatically displayed when entering the details without changing anything[/p]
• [p]Fix: Char Name synchronization between views[/p]

[p][/p][p]Time to celebrate! Explore all the features and join the fun experimenting with technology.[/p][p]The FAQ is live with extra details to help set your expectations. Take a look before you jump in and let the adventure begin![/p][p]And don't forget to share your thoughts and feature requests in the Community Hub![/p][p][/p][p][/p]

[p]Hey all, one last thing for this year and an addition to the last Devlog! We finally have a release date for AI Pals Engine on Steam: January, 8th! [/p][p]Of course 2026, I promise! If you don't have it already, remember to wishlist to get notified![/p][p]Wishing you all Happy Holidays! Thank you for all, can't wait to see your Widgets![/p][p][/p]

[p]Hey everyone! In Devlog #0, we discussed why AI Pals Engine was created. Now, in this follow-up, we're diving into one of its core features: Desktop Widgets.[/p][p]Discover what they are, explore the types you can use, and see how they'll transform your PC into a truly personal space - interactive and uniquely yours. Let’s get into it![/p][p][c]Bring Your Desktop to Life[/c][/p][p][/p][p]You know them as small, handy tools that sit right on your desktop, giving you quick access to what matters to you. Instant info, right where you need it.[/p][p]In AI Pals Engine, widgets are not just simple panels - they’re dynamic containers for customizable content panels, letting you pick exactly what you want to see and use. All organized within a switchable Widget Space.[/p][p]With AI Pals Engine Widgets, you can:[/p][p]✔️ Place them anywhere on your screen[/p][p]✔️ Fully customize their look, function and behavior[/p][p]✔️ Use them as floating overlays (Foreground Mode) with or without Mouse Interaction [/p][p]They’re your personal desktop extensions - flexible and built to fit your needs.[/p][p][/p][p][/p][p]Widgets are designed around your AI Pal Avatar - whether you're going lightweight (pure web tech based), 2D (using Phaser) or full 3D (Three.js). Or maybe you don't want a Pal Avatar at all! The choice is yours. Plus, there's a Webpage Render Widget - great for embedding external content, like dashboards or live feeds. And with shortcut support, you can toggle them on and off instantly.[/p][p]No bloat. No forced features. Just plug in the widgets that fit your vibe or experiment with the pre-built examples. And let's add some cool new Widgets during the Early Access![/p][p]While Widgets are a type of container, Panels and Pals are the dynamic components that bring them to life - custom building blocks you place directly into your widgets.[/p][p]Pals [/p][p]Your AI Pal, OPTIONALLY powered by a text model and/or represented by a visual avatar like images, shaders, spritesheets, videos or 3D models - all static or behavior-driven (more on that in a future devlog).[/p][p]Panels [/p][p]These are the functional building blocks - everything not a Pal. Here's what you can use:[/p]

• [p]Text Panels: Go beyond static text. Display time, date, CPU usage... or dynamic content using dynamic tokens (explained in a future devlog).[/p]
• [p]Action Buttons: Click them to trigger actions[/p]
  • [p]App Actions: Minimize the widget, trigger the sidepanel and more... bring frequently used features out front.[/p]
  • [p]Manual Tool Actions: Run specialized tasks on your machine or externally (just like your Pal can trigger them).[/p]
  • [p]Transformation Actions: Perform quick, non-chat tasks with one click - proofread, format, summarize or manipulate text in any way.[/p]
• [p]Note Panel: Your flexible workspace for long-form content. Supports Markdown/HTML, perfect for notes, todos or live info display.[/p]
• [p]AI Panels: Input & output panels for your text AI, with multiple variants to match your needs.[/p]

[p]With the panels, you can use your Desktop in a new way and also have the flexibility to create Interchangeable Environments (e.g. different roleplay spaces).[/p][p] [/p][p]It's a reimagining of what your PC can be. The more you explore the App and get comfortable with its features, the more amazing things you can create. Don't want to dive too deep? No problem, there are ready made example widgets.[/p]

• [p]Turn your screen into a living space: Display what matters most to you... real-time stats, AI-generated text, personal notes or even live feeds.[/p]
• [p]Add your own workflows: Time, buttons, AI inputs and outputs... all can trigger full sequences of actions.[/p]
• [p]Blend style and function: Make your desktop reflect your mood and vibe. Switch to a work-focused widget space when you need to focus or a roleplay setup when it's time to have fun.[/p]
• [p]Share and grow: Export, import and share widgets with others. Discover new setups, inspire others, or create something so unique it becomes a standard.[/p]

[p]We explored the WHY in the first devlog: AI Pals Engine is designed as a highly customizable platform for diverse tasks. The powerful flexibility of Widgets and Behaviors makes this possible. However, it's important to remember that AI Pals Engine isn't a pre-built experience tailored to a single purpose; instead, it's a sandbox where you select and integrate the components you need.[/p][p]To leverage more complex features, some investment in learning the underlying concepts is required. For example, the included RSS example demonstrates functionality, but understanding how Behaviors operate is essential for integrating it into your own spaces.[/p][p]Currently, Themes and Styling aren't deeply integrated with all elements, such as the General Text panel. Styling often requires direct CSS/Stylesheet manipulation, though it is planned to improve this over time, potentially incorporating AI-driven UI generation.[/p][p]Regarding performance: Do you need a high-end PC? It depends on how you intend to use it! Simple AI pals that are used purely for decoration can run on a Potato. The standard AI Model that is included will also do its job in a reasonable timeframe, even on toaster to mid-range setups. However, utilizing the latest, state-of-the-art large AI models will demand a rig that is beyond your setup. A mid-range AI setup with 3D Pals should perform well on a high-end gaming PC. Fortunately, AI Pals Engine offers energy management and optimization options. For instance, widgets default to pausing after 14 seconds of inactivity and you can minimize them when they aren't needed.[/p][p][/p][p]AI Pals Engine handles all the underlying processes, including loading AI models (even simultaneously), text-to-speech, rendering, Behaviors, chat history and more. This complexity is managed internally and if you find any features or options missing, let me know - that's the benefit of the Early Access Release![/p][p]What kind of Widgets would you like to see? There's so much room to get wild and use the EA to build new Widgets alone. Can't wait to share more with you! In the end, it's not just about the visuals. The behavior matters too and that deserves its own Devlog. If you're interested in participating a test before the EA Release, please don't hesitate to send me a DM on Discord (or contact me directly somewhere else) for more details.[/p][p]Thanks for reading and supporting this journey. Have an awesome weekend![/p]