DFHack - Dwarf Fortress Modding Engine News

This update works with DF 50.09 on both the Dwarf Fortress default and beta branches. The DF beta branch includes native support for Linux, which this DFHack update also supports.
[h2]Linux support![/h2]
Both DF and DFHack now run natively on Linux. If you're on Linux, we encourage you to try it! The interface is noticeably snappier, FPS is up, and load times are reduced. If you're subscribed to DF and DFHack on Steam, here's how to switch over:

• In the Steam client, open the properties for Dwarf Fortress
• In the "Betas" section, switch to the "beta" branch
• Let that finish updating
• Select the "Compatibility" section and deselect "Force the use of a specific Steam Play compatibility tool"
• DF should update again with the Linux native version

You can tell if you did it right (and sometimes Steam gets stuck somewhere along the way and you have to fiddle with it) if you no longer have Dwarf Fortress.exe in your DF directory and instead have dwarfort, which is the Linux binary. The plan is to "officially" release Linux support with DF 50.10, so once DF 50.10 comes out, you should be able to switch back to the default Steam branch and keep the native Linux support.

Then, do the same thing for DFHack. switching to the "beta" branch (if you're not there already) and turning off the compatibility layer.

Major caveat for this beta release

This particular beta release of Dwarf Fortress has address layout randomization enabled. This causes some trouble for DFHack since DFHack needs to know the memory addresses of several key data structures. For this release only, be sure to start DF by launching DFHack from the Steam client or by running the ./dfhack commandline startup script. That will normalize the address space and allow DF to run with DFHack. Otherwise, DF will crash on startup. Again, this is only for this release. Future releases will not need this workaround and should go back to allowing you to start DF from either the Dwarf Fortress or DFHack Steam client entries, or from either the ./dwarfort or ./dfhack commandline commands.

The DFHack terminal console works differently on Linux

You can run DF with DFHack by starting DFHack in the Steam client. However, if you want an external DFHack terminal console, you have to run from the commandline.

On Windows, you could use the show command to pop up an external DFHack terminal console. You could use this for running DFHack commands from outside the game window, and the external terminal is the only way to run commandline-interactive DFHack commands like tiletypes and the interactive mode of the lua interpreter.

Linux has a different method of providing a terminal console. You can't spawn it dynamically like you can on Windows. You have to start DF from the commandline, and the terminal from which you ran ./dwarfort becomes the terminal console. This should be a familiar process to many Linux users (most Linux commands work this way), but the change is jarring if you're not expecting it. We're also looking into providing a virtual console that doesn't depend on an existing system console so you can still get a console even if you run from Steam, but that work is far from being completed.

You can still launch DFHack from Steam if you want to. Many tools log information and errors to the console, though, so if you run into strange issues, it might be useful to try running from the commandline to see if there is diagnostic output there that can help you.
[h2]Search and sort for squad assignment screen[/h2]
When you bring up the screen to assign units to squads, there are now widgets for searching and sorting. The default sort is by "best melee skill", but you can choose from a variety of relevant ascending or descending sorting orders.

You can also search. Searching matches either the unit name or a skill that the unit has. Sorting by "leadership" shows units that have any leadership-related abilities, like teaching and military tactics training, but if you just want to see dwarves with military tactics training, search for "tactics".

There have already been many good suggestions for additional sorting options, which we'll try to get into a future beta/release.
[h2]Animal assignment[/h2]
The "Assign to pasture" screen released in the previous DFHack version has been expanded and generalized to support cages, restraints, and pits/ponds. Please tell us if you have any additional needs for animal assignment that those screens don't meet!
[h2]One-click UI integration for exportlegends[/h2]
exportlegends now sports a new integration with the vanilla "Export XML" button. Now you can generate both the vanilla export and the extended data export with a single click!
[h2]Onscreen HUD getting in your way? Hide it![/h2]
hide-interface hides all the vanilla UI elements for clean screenshots or distraction-free fortress watching. Even with the interface hidden, you can still pause/unpause the game with spacebar and move around the map with the keyboard or mouse. Hide that cluster of urgent notifications bubbles for a while and just enjoy watching your citizens scurry around : )
[h2]New Tools[/h2]

• devel/scan-vtables: Scan and dump likely vtable addresses (for memory research)
• hide-interface: hide the vanilla UI elements for clean screenshots or laid-back fortress observing

[h2]New Features[/h2]

• exportlegends: new overlay that integrates with the vanilla "Export XML" button. Now you can generate both the vanilla export and the extended data export with a single click!
• sort: search and sort for squad assignment screen
• zone: advanced unit assignment screens for cages, restraints, and pits/ponds

[h2]Fixes[/h2]

• Core:
  • reload scripts in mods when a world is unloaded and immediately loaded again
  • fix text getting added to DFHack text entry widgets when Alt- or Ctrl- keys are hit
• caravan:
  • Correct price adjustment values in trade agreement details screen
  • Apply both import and export trade agreement price adjustments to items being both bought or sold to align with how vanilla DF calculates prices
• orders: prevent import/export overlay from appearing on the create workorder screen
• starvingdead: ensure sieges end properly when undead siegers starve
• suspendmanager:
  • Fix the overlay enabling/disabling suspendmanager unexpectedly
  • Improve the detection on "T" and "+" shaped high walls

[h2]Misc Improvements[/h2]

• Surround DFHack-specific UI elements with square brackets instead of red-yellow blocks for better readability
• devel/lsmem: added support for filtering by memory addresses and filenames
• hotkeys: don't display DFHack logo in legends mode since it covers up important interface elements. the Ctrl-Shift-C hotkey to bring up the menu and the mouseover hotspot still function, though.
• suspendmanager: display a different color for jobs suspended by suspendmanager

[h2]API[/h2]

• RemoteFortressReader: add a force_reload option to the GetBlockList RPC API to return blocks regardless of whether they have changed since the last request
• Items::getValue(): remove caravan_buying parameter since the identity of the selling party doesn't actually affect the item value

[h2]Lua[/h2]

• dfhack.items.getValue(): remove caravan_buying param as per C++ API change
• dfhack.screen.readTile(): now populates extended tile property fields (like top_of_text) in the returned Pen object
• new(): improved error handling so that certain errors that were previously uncatchable (creating objects with members with unknown vtables) are now catchable with pcall()
• widgets.BannerPanel: panel with distinctive border for marking DFHack UI elements on otherwise vanilla screens
• widgets.Panel: new functions to override instead of setting corresponding properties (useful when subclassing instead of just setting attributes): onDragBegin, onDragEnd, onResizeBegin, onResizeEnd

[h2]Structures[/h2]

• Added global_table global and corresponding global_table_entry type
• viewscreen_legendsst: realign structure
• viewscreen_new_arenast: added (first appeared in 50.06, probably)

This release is compatible with all distributions of Dwarf Fortress: Steam, Itch, and Classic.

• Install DFHack from Steam
• Manual install
• Quickstart guide (for players)
• Modding guide (for modders)

Please report any issues (or feature requests) on the DFHack GitHub issue tracker. When reporting issues, please upload a zip file of your savegame and a zip file of your mods directory to the cloud and add links to the GitHub issue. Make sure your files are downloadable by "everyone with the link". We need your savegame to reproduce the problem and test the fix, and we need your active mods so we can load your savegame. Issues with savegames and mods attached get fixed first!
[h2]Announcements[/h2][h3]You can now distribute your quickfort blueprints in mods![/h3]


Think you have a cool fortress design that you want to share with the community? The effort required to share your designs has now been cut down significantly. DFHack quickfort now supports blueprints that have been distributed with mods!

The blueprint in the screenshot comes from a mod with only two files:

• mods/myk002 blueprints/info.txt
• mods/myk002 blueprints/blueprints/from_mod.csv

with info.txt containing just the basic mod metadata (detailed in the DF mod guide). You can distribute these mods anyway you like, including from the DF Steam Workshop.

How do you make blueprints of your fort to distribute? DFHack's gui/blueprint will take a snapshot of your fort and create a series of blueprints for you. You can also make edits with a text editor or online spreadsheet app to do some really advanced stuff.
[h3]PSAs[/h3]
As always, remember that, just like the vanilla DF game, DFHack tools can also have bugs. It is a good idea to save often and keep backups of the forts that you care about.

Many DFHack tools that worked in previous (pre-Steam) versions of DF have not been updated yet and are marked with the "unavailable" tag in their docs. If you try to run them, they will show a warning and exit immediately. You can run the command again to override the warning (though of course the tools may not work). We make no guarantees of reliability for the tools that are marked as "unavailable".

The in-game interface for running DFHack commands (gui/launcher) will not show "unavailable" tools by default. You can still run them if you know their names, or you can turn on dev mode by hitting Ctrl-D while in gui/launcher and they will be added to the autocomplete list. Some tools do not compile yet and are not available at all, even when in dev mode.

If you see a tool complaining about the lack of a cursor, know that it's referring to the keyboard cursor (which used to be the only real option in Dwarf Fortress). You can enable the keyboard cursor by entering mining mode or selecting the dump/forbid tool and hitting Alt-K (the DFHack keybinding for toggle-kbd-cursor. We're working on making DFHack tools more mouse-aware and accessible so this step isn't necessary in the future.
[h2]Highlights[/h2][h3]Initial screens for search and sort[/h3]
The first few screens are done for search/filter/sort support! For starters, we decided to tackle the animal assignment to pasture screen and the trade screens, both of which are a frequent source of frustration for players. There is generally not enough room to add vanilla-sized buttons, so DFHack screen links will appear surrounded with red and yellow bars to to distinguish them from vanilla widgets.



The screen that comes up can be filtered and sorted by whether and where the animals are pastured/caged, by their friendliness or tame status, and by various other properties. All animals of a particular species are now listed next to each other so you can find them easily.



The trade screens got a similar treatment. There are two new screens, one for bringing trade goods to the depot, and one for doing the actual trading. In both, items that are ethically unacceptable to the traders that you are trading with and items that are forbidden to export by your nobles are automatically filtered out, though you can change the filters if you like. Both screens also make bins safe and easy to work with.





The bring goods to depot screen allows you to find the items that you want to sell and mark them for trading. For example, you can move the "condition" slider to only show damaged items and then mark them all for trade with a single click.



The trade screen provides similar filter and selection support for items being bought and sold. By default, only the contents of bins are listed and traded and the bins themselves aren't traded. You can toggle this behavior and trade bins if you like, though.



[h3]Copy/paste support[/h3]
With the move to SDL2 comes new clipboard integration capabilities. DFHack's text entry fields, such as the command editing field in gui/launcher, now support copy, paste, and cut. Ctrl-V will paste from the system clipboard into the DFHack text field. This is especially useful when pasting in DFHack commands that you find on the internet! Ctrl-C will copy text out that you can paste into another application, and Ctrl-X will copy the text and also clear the field (so it's a convenient way of just clearing the text quickly even if you don't need to copy it).

Note that there is no way to just select a portion of the text, though, so all copy and cut operations apply to all text in the field.



[h3]Warm and damp indicators in ASCII mode[/h3]
DF does not distinguish damp or warm tiles when in ASCII mode. This can be very frustrating for ASCII-mode players that have to play without this information.

DFHack now automatically highlights damp tiles in blue and warm tiles in red when in ASCII mode and a mining operation is selected. In other words, the visual feedback in ASCII now matches what premium players get in graphics mode.





[h3]Other new tools[/h3]
Some long-standing requests were finally fulfilled in this release, with the return of a few fan-favorite tools and one new game fix.

• 3dveins replaces vanilla DF's blobby vein generation with veins that flow smoothly and naturally between z-levels. Want a more natural looking geology? Try running this command right after a new embark!
• dwarfvet allows your animals to have their wounds treated at hospitals, prolonging their life and usefulness.
• fix/empty-wheelbarrows dislodges rocks that get stuck in your wheelbarrows when a hauling dwarf gets distracted and the full wheelbarrow gets abandoned somewhere. This makes those wheelbarrows useful for hauling again. Enable this tool in the "Maintenance" tab of gui/control-panel to automatically keep your wheelbarrows usable!

[h2]New Tools[/h2]

• 3dveins: reinstated for v50, this plugin replaces vanilla DF's blobby vein generation with veins that flow smoothly and naturally between z-levels
• caravan: new trade screen UI replacements for bringing goods to trade depot and trading
• dig: new dig.asciiwarmdamp overlay that highlights warm and damp tiles when in ASCII mode. there is no effect in graphics mode since the tiles are already highlighted there
• dwarfvet: reinstated and updated for v50's new hospital mechanics; allow your animals to have their wounds treated at hospitals
• fix/empty-wheelbarrows: new script to empty stuck rocks from all wheelbarrows on the map
• zone: new searchable, sortable, filterable screen for assigning units to pastures

[h2]Fixes[/h2]

• Fix extra keys appearing in DFHack text boxes when shift (or any other modifier) is released before the other key you were pressing
• gui/autodump: when "include items claimed by jobs" is on, actually cancel the job so the item can be teleported
• gui/create-item: when choosing a citizen to create the chosen items, avoid choosing a dead citizen
• gui/gm-unit: fix commandline processing when a unit id is specified
• logistics:
  • don't autotrain domestic animals brought by invaders (they'll get attacked by friendly creatures as soon as you let them out of their cage)
  • don't bring trade goods to depot if the only caravans present are tribute caravans
  • fix potential crash when removing stockpiles or turning off stockpile features
• suspendmanager:
  • take in account already built blocking buildings
  • don't consider tree branches as a suitable access path to a building

[h2]Misc Improvements[/h2]

• Dreamfort: give noble suites double-thick walls and add apartment doors
• Suppress DF keyboard events when a DFHack keybinding is matched. This prevents, for example, a backtick from appearing in a textbox as text when you launch gui/launcher from the backtick keybinding.
• autonick: add more variety to nicknames based on famous literary dwarves
• gui/unit-syndromes: make lists searchable
• logistics: bring an autotraded bin to the depot if any item inside is tradeable instead of marking all items within the bin as untradeable if any individual item is untradeable
• quickfort: blueprint libraries are now moddable -- add a blueprints/ directory to your mod and they'll show up in quickfort and gui/quickfort!
• stockpiles: include exotic pets in the "tameable" filter
• suspendmanager: display the suspension reason when viewing a suspended building
• widgets.EditField: DFHack edit fields now support cut/copy/paste with the system clipboard with Ctrl-X/Ctrl-C/Ctrl-V

[h2]API[/h2]

• Items::markForTrade(), Items::isRequestedTradeGood(), Items::getValue: see Lua notes below
• Units::getUnitByNobleRole, Units::getUnitsByNobleRole: unit lookup API by role

[h2]Internals[/h2]

• Price calculations fixed for many item types

[h2]Lua[/h2]

• dfhack.items.getValue: gained optional caravan and caravan_buying parameters for prices that take trader races and agreements into account
• dfhack.items.isRequestedTradeGood: discover whether an item is named in a trade agreement with an active caravan
• dfhack.items.markForTrade: mark items for trade
• dfhack.units.getUnitByNobleRole, dfhack.units.getUnitsByNobleRole: unit lookup API by role
• widgets.TextButton: wraps a HotkeyLabel and decorates it to look more like a button

[h2]Structures[/h2]

• build_req_choicest: realign structure and fix vmethods
• squad_orderst: fix vmethods

[h2]Documentation[/h2]

• misery: rewrite the documentation to clarify the actual effects of the plugin

Initial bugfixes for the new trading screens:

• Trade screen now refuses to launch if the traders are not ready to trade
  
• Trade screen now initializes correctly when the traders
• are* ready to trade
  
• Items in buildings are now correctly marked for trade when selected in the "Bring trade goods to depot" screen
  

The focus of this beta release is the trade workflow. There are two new DFHack screens to play with, one for selecting items to bring to the depot and one for selecting caravan and fort items for the actual trade transaction.

We need your feedback on these! Are they easy to use? Can you get all the items you want to the trade depot in a "reasonable" number of clicks? Are there more filters you'd like to see?

Note that while the "Bring goods to depot" screen is separate from the vanilla screen, the "Trade" screen can be used simultaneously with the vanilla interface. The checkboxes on the vanilla screen are updated "live" as you use the DFHack selection interface, and vice versa. Also, the vanilla screen trade summary at the bottom that shows whether the trader is likely to accept the deal is updated in realtime.

The DFHack "Bring goods to depot" screen is accessible by clicking on the text button that is just above the vanilla "Bring goods to depot" button (shown when you click on the trade depot). It also has the hotkey Ctrl-T

The DFHack trade screen is accessible from the vanilla trade screen, again by clicking on the text button on the right side of the screen or by pressing Ctrl-T.

Hopefully the screencaps attached to this post make everything clear. Suggestions for further usability improvements are always welcome!

Getting these screens done was truly a team effort, requiring significant reverse engineering for information on how export agreements affect price calculations, the logic behind export mandates, and the details regarding ethical trade restrictions, plus thousands of lines of business logic. Very heartfelt thanks to DFHack resident reverse engineers Quietust and ab9rf (rome of oxtrot), without whom none of this would have been possible!










Now that DF is on SDL2, we get much better integration with operating system services. Edit fields, like the commandline text line in gui/launcher, now supports copy and paste. Ctrl-C to copy, Ctrl-X to cut, and Ctrl-V to paste. This means that you can find a sample commandline online, open up gui/launcher, and hit Ctrl-V to paste it in. No more manually copying it word by word!

We don't have a way to select ranges of text, so Ctrl-C will copy the entire line and Ctrl-X will cut the entire line. On the plus side, this means Ctrl-X is now a convenient way to clear any DFHack text field : )
[h2]New Plugins[/h2]

• 3dveins: reinstated for v50, this plugin replaces vanilla DF's blobby vein generation with veins that flow smoothly and naturally between z-levels

[h2]New Scripts[/h2]

• caravan: new trade screen UI replacements for bringing goods to trade depot and trading

[h2]Fixes[/h2]

• Fix extra keys appearing in DFHack text boxes when shift (or any other modifier) is released before the other key you were pressing
• gui/autodump: when "include items claimed by jobs" is on, actually cancel the job so the item can be teleported
• gui/gm-unit: fix commandline processing when a unit id is specified
• suspendmanager: take in account already built blocking buildings

[h2]Misc Improvements[/h2]

• widgets.EditField: DFHack edit fields now support cut/copy/paste with the system clipboard with Ctrl-X/Ctrl-C/Ctrl-V

[h2]API[/h2]

• Items :: markForTrade(), Items :: isRequestedTradeGood(), Items :: getValue: see Lua notes below
• Units :: getUnitByNobleRole, Units :: getUnitsByNobleRole: unit lookup API by role

[h2]Internals[/h2]

• Price calculations fixed for many item types

[h2]Lua[/h2]

• dfhack.items.getValue: gained optional caravan and caravan_buying parameters for prices that take trader races and agreements into account
• dfhack.items.isRequestedTradeGood: discover whether an item is named in a trade agreement with an active caravan
• dfhack.items.markForTrade: mark items for trade
• dfhack.units.getUnitByNobleRole, dfhack.units.getUnitsByNobleRole: unit lookup API by role

This release is compatible with all distributions of Dwarf Fortress: Steam, Itch (note: as of this announcement, Bay12 has not yet updated their store on itch), and Classic.

• Install DFHack from Steam
• Manual install
• Quickstart guide (for players)
• Modding guide (for modders)

Please report any issues (or feature requests) on the DFHack GitHub issue tracker. When reporting issues, please upload a zip file of your savegame and a zip file of your mods directory to the cloud and add links to the GitHub issue. Make sure your files are downloadable by "everyone with the link". We need your savegame to reproduce the problem and test the fix, and we need your active mods so we can load your savegame. Issues with savegames and mods attached get fixed first!
[h2]Announcements[/h2][h3]PSAs[/h3]
As always, remember that, just like the vanilla DF game, DFHack tools can also have bugs. It is a good idea to save often and keep backups of the forts that you care about.

Many DFHack tools that worked in previous (pre-Steam) versions of DF have not been updated yet and are marked with the "unavailable" tag in their docs. If you try to run them, they will show a warning and exit immediately. You can run the command again to override the warning (though of course the tools may not work). We make no guarantees of reliability for the tools that are marked as "unavailable".

The in-game interface for running DFHack commands (gui/launcher) will not show "unavailable" tools by default. You can still run them if you know their names, or you can turn on dev mode by hitting Ctrl-D while in gui/launcher and they will be added to the autocomplete list. Some tools do not compile yet and are not available at all, even when in dev mode.

If you see a tool complaining about the lack of a cursor, know that it's referring to the keyboard cursor (which used to be the only real option in Dwarf Fortress). You can enable the keyboard cursor by entering mining mode or selecting the dump/forbid tool and hitting Alt-K (the DFHack keybinding for toggle-kbd-cursor. We're working on making DFHack tools more mouse-aware and accessible so this step isn't necessary in the future.

[h2]Highlights[/h2]
Most changes in this release have to do with migrating to SDL2 and are not visible to the player.

[h2]Misc Improvements[/h2]

• `caravan`: new overlay for selecting all/none on trade request screen
• `suspendmanager`: don't suspend constructions that are built over open space

[h2]Structures[/h2]

• ``tiletype_shape``: changed RAMP_TOP and ENDLESS_PIT to not walkable to reflect how scripts actually need these types to be treated