Factorio News

Read this post on our website.


We recently bought and assembled some high-end PCs, with the hope to gauge the performance, speed up running tests, and potentially consolidate the number of servers we are maintaining internally. The two lucky CPUs were a i9-9980XE 18-core and a Ryzen 3900X 12-core.



We are using the time to complete our test suite in 'heavy mode' as a benchmark. Heavy mode basically saves and reloads the game each tick, and compares a CRC of the map from before and after. It is super slow to run, but the heavy test is critical to help find any possible determinism issues. There is some more info on 'heavy mode' in FFF-63.

As a baseline, the 'standard' CPU in the office for developers is the i9-7900x 10-core, which runs heavy tests in about 530 seconds. In real time this is 8 minutes and 50 seconds, a long time for a team member to sit around for results before they can push. We can do better!

As you would expect, the new 18-core was blazing fast, with a test time of about 400 seconds, shaving off over 2 minutes. However the Ryzen was a different story, with a test time of about 600 seconds. This goes against what we predicted, where more cores and higher frequency mean lower test times. The initial results from the 12-core Ryzen were worse than from the 10-core Intel; not a good start.

So I did some digging and some research, and the answer I arrived at was RAM. When we ordered the parts, not much thought was given to the selection of RAM, just some standard 16GB 2666MHz sticks to fill all the slots. Luckily, I looked on a local Czech website, and they had some stock of the brand new G.SKILL 3600MHz Trident RGB Neo, a high performance RAM stick made exactly to suit our new Ryzen CPU. After installing the new RAM, we had a test result that better matched our expectations: 450 seconds. We knew beforehand that Ryzen liked fast RAM, but we didn't realize how significant of a difference it could make.

So now we have set up both these new machines to run tests automatically after each commit, and we are very happy with the result. The new i9-9980XE can compile and run heavy tests faster than our old i7-4790K can compile and run just normal tests. Having it run automatically also frees up individual developers from the responsibility of running heavy tests locally, so they can just push as normal and continue working.


Alongside building the new servers, we also had some new storage shelves installed in the server room. So this week I spent some time moving the servers from their old home (of the floor) onto the new racks, and moving a lot of the 'PC junk' (cables, mice, fans, spidertron, keyboards, SSDs, headphones etc.) that had accumulated around the office into the server room.



We now have 10 servers, some run tests, some do the deployments, etc. We are hoping to reduce this number in the future, as we can have 1 beefy PC do the work that is currently handled by multiple older/slower ones.


This week I spotted a Reddit post by /u/Bladjomir. It is a video of this modded base after he had respawned all the biter bases. I was really in awe at just how different the game can 'feel' once you started adding and combining mods (especially when it is coupled with the classic Red Alert 2 soundtrack). So I messaged Bladjomir asking if I could include his video in this Friday facts, and he took the time to create a brand new video for us to showcase:

https://youtu.be/Qekvxw9t7zo

As someone who grew up playing the old school RTS games, these videos really hit on some nostalgia for me. What I am really proud of, on behalf of the team here, is just how well the game supports mods. It really is just so amazing to me that we have a engine now that can be extended and overhauled by mods and just works.

There will be a time when Factorio base game is finished, when we have made the game we set out to make. Yet even though our vision is complete, anybody can come along and implement their own ideas, rework what we have done, and even (theoretically) turn off the base mod and make essentially their own new game in the engine. So in the (super) long run, I believe that mod support will be critical to keep the game and community alive, and continuing to improve the modding capabilities of the game is a strong possibility for post 1.0 support/updates.

As always, let us know what you think on our forum.

Read this post on our website.

Hello,
technically this post is the Pi Friday Facts, but unfortunately we can't think of anything special to do... maybe someone can make a combinator cake... that can calculate Pi?


We released 0.17.69 as stable this Tuesday. It seems it all went very smoothly, no avalanche of crashes, and only a handful of technical support emails about updating video drivers.



Apart from stable, essentially no development work has happened this week; nearly everyone is on vacation (I am even writing this as I sit in the airport waiting to fly to London for a wedding). We're hoping that at the start of next week, with all the relaxation over, and the pressure of stable off our shoulders, we will get cracking on the next updates with renewed energy.

In fact I might be a little optimistic in saying this, but I think we are in for some exciting times here in the team. Before now, we have always done a cycle of having the whole team on development for a few months, then the whole team bug fixing for a few months. This binary approach is what gives us the traditional 'stable' and 'experimental' labels. This is not to say that all bug-fixing would stop once stable it out, quite the contrary, but this has been the general strategy.

What we are planning, if the logistics of it turn out okay, is to have significantly smaller feature releases, containing only a handful of new features. This is to have a sort of mixed cycle, and a mixed cycle in two similar ways:

• Some developers will be on bug-fixing while others are on development.
• The individual weekly/monthly work of a developer will have a more balanced mix of development and bug-fixing.

For example, while one developer works on a feature for the next feature release, another will be bug-fixing the features in the current release. This is only practically possible if the feature release frequency is relatively high.

This new structure has been a long time brewing in our minds, and we think now is the right time to try it out. With the GUIs we really need to do quick iterations and receive fast feedback to the changes. The traditional release flow meant that we could add a new feature to the game, and players wouldn't get their hands on it for months. Then once it is released and we start getting feedback on it, extra time is spent just re-orienting ourselves to the code and how it was written. If the feedback is given expediently, the rework and improvement is much more efficient.

Furthermore, I think it is more psychologically effective to work on a mix of bug-fixing and development. This is just theory now, but grounded in some observations I have made over time.

Development work, a new feature, new GUI, etc. is generally a long-form creative process. New systems have to be created out of pure thought-matter, ideas for implementation have to be evaluated and determined, and it also involves a lot of 'background processing'. Feature development always has more room for extension, and it is very hard to say 'It is finished'. It is also quite a subjective result, so sometimes it is hard to know if you 'did a good job'.

Bug-fixing on the other hand is very short form and challenge oriented. It is like investigating a murder mystery, and really feels like a complete story. Tracking down a problem inside the game engine engages a logical part of your brain, trying to piece together and backtrace where the fault is occurring. Generally the bug has a very clear 'win-condition', and you can close the game and let your mind rest peacefully. The result of a bug-fix is grounded strictly in objective measurements, so you can be reasonably sure if you 'did a good job'.

So these two parts of the job are in a way, quite distinct and separate: Development is a long-form creative process; Bug-fixing is a short-term logical process. From all this, my reasoning is that focusing on only one for a long period of time leads to quicker mental fatigue, and that a balanced workload will keep us happier and more productive.
In essence, doing development lets our bug-fix circuits rest, and doing bugs lets our development battery recharge.

There are also some pragmatic reasons I think the smaller/quicker releases will make things move along more smoothly:

• Bug-fixes after stable will be released within a short time-frame.
• The flow of bugs coming in will be less extreme, no more massive waves with each major release.
• There will be less 'blocking', where unfinished features delay a release. They will just be scheduled for a different release.
• Feedback will be more focused, so it is easier for us to evaluate.

At the start of the year, I read a book called "Rolling Rocks Downhill" by Clarke Ching. It is a book about software project management, it was quite an enjoyable read, and gave me a lot of inspiration to try and optimize our development effort. At that time we were just wrapping up 0.17, so there wasn't a whole lot of room to make changes to the way we do things. Now that we have stabilized 0.17, and with it, completed the 'traditional' cycle, there is opportunity for a fresh approach. I guess I will give the book another read next week...


As you might know, I've spent quite a considerable amount of time playing World of Warcraft classic the past few weeks. I find it very interesting that many people (me included) find this version of the game way better than the modern version, there are many reasons for it that I see (and some that I don't), yet it deserves some small analysis as it is always better to learn from mistakes of others. By that I mean, that I wouldn't want to spend next 10 years working on better versions of Factorio just to find out, that the old ones were better for some reason.

The topic that connects it all is "Making things more convenient isn't always making things better". There are a lot of activities in life and games that we would like to remove or make more convenient, but we don't really see the hidden benefits of it. Something like making an elevator to save time and effort, and feeling bad for moving less in the long run.

One of the things in wow is nicely illustrated by in this video. When people had to gather group for a dungeon back in the days (and now in classic), they had to do it manually. They had to spam LFG (*Looking for group) channels, ask in guilds or find long-time buddies to play with. In modern wow, they made a dungeon finder that allows you to just select a role and automatically teleport you to a dungeon with the appropriate people. It seems like a great convenience. But in fact, it removes so much that it is considered to be one of the most hated feature by many people.

• When it is harder to find a group, it has also more value when the group is already assembled. This means, that people are more likely to try to overcome problems while fighting rather then just quitting and re-rolling with someone else. This make it much more interesting as it created a lot of challenges where non-optimal setup or a noob in a group forced people to learn from each other more, or just to try to overcome the problems in a creative ways.
• When you are not teleported to the dungeon and have to actually travel there, it keeps the immersion of the game, the world is there to be explored, not just an arcade where you push and play. And same as with previous point, getting there requires more effort, so the value of doing the dungeon is higher.
• And generally, having more dungeon runs in the same time doesn't make it feel like you accomplished more in the game, as it generally just lowers the value of one run. And all this for the cost of it all being way more mechanical.

Another big example is level scaling. The idea is again nice, to make it more convenient. With level scaling, monsters conveniently scale to your level in the zone where you are, so you can quest at any place without having to avoid too high or low levels areas relative to your level.. But I consider this to be actually plague of any game where it appears. Instead of zones, their levels and your level meaning something, and the progress being clearly visible every level, suddenly it all disappears. The world isn't a place to explore, but rather arcade-like place that all resolves around you. Your level doesn't mean much, as it being too low isn't a problem anymore. The moments of having to run from areas where you are too weak to get back later are one of the most important in RPGs, and suddenly they are gone. The progress you see in games is like looking from a window of a train, if the train moved and the landscape moved the same way, you wouldn't feel like you are moving at all.

There are much more lessons learned while playing the game. Not only the mistakes made in modern version, but also lessens of good game design obviously. How could these be related to Factorio? I believe that on the deeper level, there could be a lot of parallels.

• Getting first mount at level 40 for 100 gold? You have to walk a lot before you get it and it might feel annoying at times. But when you actually get it, it means A LOT. Similar as how it is a game-changer to get construction robots/logistic robots/power armor. Getting there isn't easy, and there are certainly a lot of moments where you/we feel like having these sooner might be more convenient. But it would just decrease how great and valuable upgrade it is.
• We were thinking about a belt building tool, similar to the rail building tool. It would just connects ends, and even find underground connections etc to get to the point of destination. It would be super convenient. But suddenly, solving the belt puzzle wouldn't mean much, and having the belt factory would feel like much smaller accomplishment.

Anyway, thanks to all of you for such a great year so far, thanks to all our friends on the forum and throughout the community who have helped us in the great bug war of 0.17, and as always, let us know what you think on our forum.

Read this post on our website.


It has been 6 months and nearly 70 releases since we first launched 0.17 to the world. Now is the time to let is be enjoyed by all the players of the game. We are going to be continuing our work on 0.17 over the next few months, with small experimental releases of new features, and finishing all the GUI reworks.


The new map editor can be accessed directly in-game with the /editor command. Packed with new features and improvements, such as cloning tools, multiple surface support, and time control. The new editor also allows collaborative map editing in multiplayer. More details: FFF-252.


The enemies in the game, the native inhabitants of the planet, have received a major facelift in this update. As well as a graphical overhaul, the attacks and balancing of the spitters and worms has been revised, incorporating skill-based elements and some tactical decision making. More details: FFF-268, FFF-279.


Now when trying to join your favorite server, you can directly sync your mods with the server from in-game, and automatically join once they are all installed. This massively reduces the friction for joining modded servers. The in-game mod browser has also been redone to help find new mods and manage your installations.


We have massively optimised (and parallelized) the fluid update in 0.17. In effect each section of pipe can be updated independently of other pipe sections. This comes as welcome news for any megabase planners out there. Furthermore we have limited each pipe section to only a single fluid, and will actively block any placement that could mix fluids, to prevent the catastrophe of mixing water into your oil system and bricking the whole system. More details: FFF-271.


This release has a completely new rendering backend, written almost from scratch to our exact requirements. The new engine takes advantage of more modern GPU features and is highly optimized. What this means is that the game can now work with much less VRAM and allow players with older videos cards to still enjoy the game in high resolution at 60 FPS. More details: FFF-251, FFF-264, FFF-281.

Due to the more modern architecture, some issues may be present with certain hardware configurations, please see this forum thread if you experience any graphical problems.


We have added a completely new introduction scenario on a hand crafted map. It gives new players some simple goals and provides a place for them to experiment with a smaller toolset than Freeplay allows. The scenario is open ended, so that players may experiment with larger bases. A separate techtree is provided for this purpose which includes some infinite research. Our companion robot, Compilatron, is also present to help guide the player. More details: FFF-262, FFF-306.


We have added some powerful shortcut keys, which use the intuitive layout we are all used to. Now you can undo your last action by pressing CTRL + Z, quick equip a cut tool with CTRL + X, do quick copy/paste actions with CTRL + C/CTRL + V, and assign robots to upgrade your factory with the new Upgrade planner. More details: FFF-255.


We added a new GUI skin, along with the first batch of GUI reworks. The new GUI aims to unify the look and feel of the game, and provide a consistent visual basis for the players interactions with the world. More details: FFF-243.


As always there are a ton of smaller features and changes to discover in this latest major update. You can read the changelog in game for a full rundown, or checkout the release notes here.

We have our plan moving forward for the next version of the game, which you can check out here.

Read this post on our website.

Hello,
We have had quite a peaceful week here in the office.

Last night we went out for a burger and a beer as a last meal with Ernestas before he flys back home for another few months. It was also quite nostalgic, as we went to a place that we used to go to frequently near our old office, and the staff still remembered us.


It seems our 'news' of some programmers taking a break made headlines this last week: Work slows on Factorio while a dev “levels his priest” in WoW Classic.
Even though he is still not level 60, Kovarex found some time to fix a few more bugs, which seemed to go very smoothly, as he is feeling super recharged.

We also had Boskid come to visit the office, and he helped us do some intensive testing and test writing. Working together with Dominik, the two managed to find and fix a lot of bugs related to the fluid mixing system. It was certainly more efficient having someone doing QA inside of the office, so it seems our team might be getting a bit bigger.

With the last 'blocking bugs' resolved, the bug war is almost at a close. The count on the forum is down to only 16, and the automatic crash reports are looking okay:



We have released version 0.17.69, we are optimistic that this version may be able to be called stable. While this is almost over 1 month after our first 'stable candidate', we are confident now that this version is more stable than 0.16.


In last week's FFF we presented the new landfill terrain. We felt relieved by having another task done in GFX.
One of the first reactions in our forums was the idea by eradicator of rotating the grid to 45°. It followed a fast mockup made by ThreePounds. The idea was good, and apart from us, some people on the forums agreed.

The modification of our sources to get this rotation done was very affordable to us, so we decided to make it official and here we are now presenting it:



This solution is nicer than the 90°'s one because basically it doesn't interfere with the tile grid of the game, and provides a nice loose space feeling in the general composition. I just felt a bit guilty for not coming with the best approach from the beginning, but luckily our community is very keen to give us feedback on any aspect of the game.

Thanks guys for the feedback.

As always, let us know what you think on our forum.

Read this post on our website.


Hello Factorians,

Today I would like to talk to you about my favourite subject - fluid mixing and its prevention. It is a new mechanic introduced in 0.17 that seemed quite simple at first, but has been giving me nightmares ever since.

A while ago I took on the task of updating the fluid system (FFF-260). The way it works in 0.16 is that the fluid boxes (the thing that holds the fluid and is contained in entities like pipes or refineries) had no organisation whatsoever except their connections. They would sit there and do nothing and then once per update send fluid somewhere. It had it's issues especially with symmetry, and when you happened to put some fluid where it did not belong, you could end up requiring major demolition works.

My task was to develop a better algorithm to move the fluids, and a very very optional side task I had in mind was to do something about the fluids getting to wrong places and mixing. We started by organizing the fluid boxes into systems (connected FBs make a system) managed by a special fluid manager, and optimizing the heck out of it (FFF-271).

Soon after, I started working on the new algorithm and anti fluid-mixing. Until then nobody really considered preventing the fluid mixing seriously as it did not seem possible. But when I thought about it, it did not seem that hard. The idea is to simply not allow any action that would lead to fluids getting where they are not supposed to go. When you build a steam pipeline, you should not need nor want it to touch another fluid.

In principle, it would only require a few quite realistic mechanisms:

• A connected block of fluid boxes would either be fluid-free, or it would be locked to one fluid.
• Two such blocks can connect only if they are compatible - either one has no fluid lock, or they have the same fluid.
• An assembler can only set a recipe if it is compatible with the fluid locks on its fluid boxes.
• Have a migration from old saves that can contain mixing.

By creating the fluid systems right before, number 1 was almost finished and we only needed to assign the lock. It would be set either by a fluid, or by a 'filter', which is a fluidbox that is set to use a fluid - such as a water pump using water, or assembler having some inputs/outputs given by a recipe.

After a while I had both the fluid algorithm and the mixing done (FFF-274). The mixing was not that easy (like 5x more complicated) but it worked pretty well. As for the fluid algorithm, V453000 and Twinsen found some issues with waves on a macro scale, and because it was right before releasing the 0.17 experimental, we decided to hold it off on it for the time being (we have a new version now that seems okay, but has to wait for 0.17 becoming stable first). The mixing made it through though and seemed quite finished.

It turned out that the work on it was at most one tenth complete.



Some difficulties appeared already before the initial release, but that was only a hint of what would come later. One by one, then ten by ten, bugs started coming. The problem is that as often as not, they were not just little issues with simple fixes. Instead, over and over, they turned the current solution on its head and the code had to be completely rewritten in order to account for the new case. As of now, almost exactly half a year and many rewrites later, it is still not fully bug free.

The number one issue was with Assembling machines. It turned out that their code was pretty messy and does not simply allow the checks I needed, so it had to be refactored. The next problem was that the check was not only required when setting a recipe on an existing assembler - as I naively thought - but in about one million different situations I would not dream existed. Building a fixed recipe assembler. Reviving an assembler. Rotating. Blueprint of an assembler. Blueprint with a recipe and a rotation placed over a ghost with a non-default rotation during a full moon. Teleport. Script building. Copy-paste settings. Any combination of these.

Pretty much every case would go through different paths in the code and behave entirely differently. Fixing one would break another and so tons of tests had to be written. When these cases finally worked, it turned out that doing these operations when they fail (e.g. can’t rotate because it would cause mixing) brings another wave of issues. And then mods come and the complexity multiplies. All this, although in a more simple form, had to be done for all other entities that can use fluid too, such as inserters (mods...).

Another number one issue are underground connections. When playing, you would not think twice about them but on a closer look they are all but simple. They have this (cough) uncomfortable feature (want to shoot myself) that you can build another underground pipe between them (or take it out) and a complex reconnection logic jumps in. It means that based on building order, connections are established and reestablished differently. This is a big thing when building a blueprint with many pipes, or loading a save game.

But the largest issue is one tiny corner case with huge implications. Have two underground pipes that have different fluids, but are split by a third, and it gets removed. Until now mixing was theoretically completely avoidable, but here it was and there was nothing to stop it.



As a result, a new concept is introduced - a blocked connection. An underground connection that would normally connect but can’t. Establishing it is the easier part. But when does it get fixed? An entity miles away gets rotated, that splits a fluid system, disconnecting the blocked connection from a fluid filter on the other side, and the connection should unblock. But even something as simple as a rotation contains fluid fixes and re-establishing of fluid boxes and if it fails, it still fixes the blocked connection in the process and it can’t be undone... You get the picture. And we are still talking about underground pipes, while anything can have underground connections, including said assemblers, which the previous code did not consider at all. The complexity just goes deeper and deeper, and Rseding is probably right that we should have never gone this way at all.

So the bug fixing has been basically running in circles - clearing one batch of bugs, thinking that those were the last ones and the ordeal is finally over, only to find another batch (ideally from some bizzare modder idea) a few days later. Many times I wished that mods never existed. Nor multiplayer, or any players at all for that matter. About two months ago all seemed really fine, with basically no reports and just a few crashes in the automated crash reports.

At that moment another nightmare materialized in the form of a talented volunteer bug tester named boskid, who took on a personal crusade to break the fluids to dust (I actually imagine him sitting in his dark room with evil laughter, dreaming about making my next day worse than the previous). In all seriousness, he has done a great deal of work with his bug testing, creating bizzare modded cases and testing scripts, and deserves a big thanks. He is actually coming to our offices next week, so follow the news for reports of developers falling out of windows.


An example of a nice configuration he found (and I can’t fix).

The whole time we were taking the offensive approach to mixing - crash the game when it happens - so that we know about bugs to fix them. Recently we have decided to put a stop to it and have the mixing automatically fix itself once it is detected, eyeing the end of this episode. Right now I have 3 mixing bugs on the list and I am sure those are the last, and mixing will be done (lying to myself is a way to cope with it) and the new fluid algorithm can come soon after :).


(And of course, boskid found a way to use the automatic mixing fixing as an evil PVP tactic.)


This week Ernestas came physically to the Wube office to spend some time with us (he is normally working remotely), so we are taking the chance to finish things that we had in the drawer that needed closer communication. One of those things is a specific terrain for landfill. As you know we were using grass for it. Not anymore.



In Factorio we have a clash of two worlds:

• The natural and organic planet. Off-grid with multiple terrain types, trees and doodads.
• The world of the factory, which is mathematical, regular, modular, and totally attached to the grid.

This new terrain represents a boundary in between those two worlds. The grid on its surface tries to express the artificiality of it, like mechanically pressed with some sort of industrial stamp. The imperfections of the grid and its irregularities are holding this man made terrain into the realm of the natural environment.

For now the sound of the landfill action is not solved, 'soon' we’ll take care of it.

As always, let us know what you think on our forum.