Dev Log - Interstellar Logistics Optimization
Interstellar logistics has always been one of the most complex systems in Dyson Sphere Program. It is not only complicated in terms of code but also in terms of design.
Many engineers have asked us this question, "Youthcat! Why does my ship go to a farther planet to get items instead of a closer one？"
Good question! In today's dev log, we will introduce the history and evolution of the interstellar logistics system. We'll roughly explain the iteration process of optimizing the interstellar logistics system experience.
[h2]Before the Update
[/h2]
In the previous interstellar logistics system, the transportation of an item from interstellar logistic station A to Station B was facilitated by a mechanism called Supply-Demand Pairing. Each supply-demand pair contained information about the supplier, the demander, and the item being transported.
Each interstellar logistic station managed all its supply-demand pairs internally. It needed to check with all other stations to see if its supply and demand could match with those of other stations, and then store all the matched supply-demand pairs.
All these supply-demand pairs formed a complex logistics network. For example, if there were 10 interstellar logistics stations in the universe demanding iron ore and 100 stations supplying iron ore, there would be 1000 supply-demand pair routes regarding iron ore.
After obtaining all the pairing information, logistic stations need to undergo periodic updates. As you all know, interstellar logistic stations are very convenient to use. You just take out items from your inventory, place them on the ground, set the items, and the goods start to be transported continuously.
Therefore, generally speaking, the number of interstellar logistic stations in the entire universe is not too small. Consequently, the pairing information within each interstellar logistic station grows exponentially.
To balance the entire supply-demand relationship (as well as optimize performance costs), each time a station judges the pairing information for picking up goods, it adopts a "fair share" strategy. This means traversing through all the pairings one by one and marking the last position after the traversal ends. The next traversal starts from this position to ensure that each pairing has a chance to be "loved".
Behind the vast transportation routes, thousands of stations and tens of thousands of supply-demand pairings perform logical computations every second.
Seeing this, you might already feel your brain getting a bit fuzzy, right? But the complexity doesn't stop here!
To avoid wasting transportation resources, if a pairing passes various logical checks (such as availability of goods, energy, transport ships, etc.), it also needs to consider whether it can conveniently deliver some additional items.
For example, when a vessel destined to pick up iron ore sets off, it needs to reevaluate if it can conveniently bring a small gift to the supplying station on its way. Similarly, when a vessel from the supplying station arrives at the destination and unloads the iron ore, it needs to check if it can conveniently pick up some other items to take back.
Our intentions were good, but things went awry during execution. Code is rigid. It doesn't care whether a pairing is closer or farther away. It simply follows the rules: if a pairing is detected and passes all logical checks in this frame, then that's the route to take!
This puts a strain on our engineers, especially when transport ships are relatively slow in the early stage. It's frustrating to watch a transport ship ignore a mining area just light-years away and instead head straight for a destination tens of light-years away, never looking back.
[h2]Beginning the Optimization[/h2]
When the CentreBrain requested optimization of the interstellar logistics system, the designers were quite unwilling, even resistant at the beginning. The previous logic was already complex enough, and further optimizing the experience would only lead to considering even more complex situations.
As the saying goes, the complexity of design doesn't decrease; it shifts. Now that we have such a complex situation causing experiential issues, the only solution is to devise even more complex rules to match these complex demands.
After continuous battles between designers and programmers, we decided to expand the "pairing" mechanism into multiple dimensions. We allowed engineers to set priorities across different dimensions, ensuring that engineers could adjust the priorities by manipulating pairings across various dimensions. This approach aimed to reduce the occurrences mentioned before and enhance engineers' sense of control over the entire cosmic factory.
We can do it!
The new design of supply-demand pairings consists of five dimensions:
① Point-to-point priority pairing between individual interstellar logistic stations.
② Setting up "routes" between planets for specific items, with priority pairings for stations along these routes.
③ Setting up "routes" between star systems for specific items, with priority pairings for stations along these routes.
④ Grouping interstellar logistic stations, with priority pairings for stations within the same group.
⑤ Regular pairings for items not assigned any specific priority.
Their priority decreases sequentially, meaning that interstellar logistic stations will first check if point-to-point pairing (①) is feasible for transportation. Then, they will consider dimensions ②, ③, and ④ in order. Finally, they will only address regular pairings (⑤) if necessary, based on the settings.
After setting up a planetary route between the third and fourth stars, titanium glass will be prioritized for transportation between these two planets.
Just as we were eagerly diving into development, we encountered a problem after a week of joyful coding — the situation was far more complex than we had imagined.
During testing, we found that in specific situations, logistic stations planned for point-to-point pairings(①) couldn't transport due to reasons like lack of warpers or energy. They could still transport (either delivering or fetching) when considering regular pairings (⑤).
After obtaining items from regular pairings, when it's time to check point-to-point pairings (①), it may be discovered that there are no items available. Alternatively, in cases where transportation is not feasible in ①, some demanders from lower-priority routes might opportunistically "steal items". Because they do not know why ① doesn't work, whether due to a lack of energy or a lack of warpers, the handling methods for these situations are different.
Moreover, the lack of energy needs to be categorized into situations where power is disconnected or not. Some situations require waiting for higher priority, while others cannot wait.
If these situations are not addressed, low-priority pairings will almost always override high-priority pairings, leading to the prioritization meaningless.
A simplified transportation model, where arrows represent transportation direction, and double arrows represent high-priority pairings.
So we adjusted the design of priorities. Priority no longer completely determines who goes first and who goes later. Moreover, in situations where high-priority pairings cannot be transported, it is not guaranteed that low-priority pairings will be served next; it depends on the specific circumstances. By studying the departure conditions of transport ships, we found that there are 9 situations to consider for each station.
Therefore, there are a total of 9x9=81 different situations to consider for both the supply and demand sides.
The designers and programmers felt "intimidated" looking at this table.
Bad news: every 1000 logistic stations will increase CPU overhead by approximately 1ms on average.
some fake codes
To implement these mechanisms, not only do we need to synchronize the timing of updates, but we also need to introduce a "locking" mechanism. When a pairing is transporting items between each other, they will apply a lock to each other. If a lower-priority pairing attempts to transport items and finds a high-priority lock on the logistic station, it will "back off".
Conversely, if a higher-priority pairing attempts to transport items and finds a low-priority lock, it will simply ignore the lock. Of course, this is just a simplified situation. More complex situations, such as when a higher-priority pairing cannot transport items, are not discussed here.
A Shipshape Panel
[h2]Thank you all[/h2]
After this update, the brand new interstellar logistics system is about to meet all the engineers.
We appreciate everyone's patience during this period without new features, and we are grateful for your continuous support and love for "Dyson Sphere Program".
As mentioned earlier, the complexity of design doesn't decrease; it shifts. However, we will always carefully consider engineers' needs and will not give up simply because of design or code complexity. In the days to come, we will continue to bring updates "Dyson Sphere Program".
Thank you all!
Many engineers have asked us this question, "Youthcat! Why does my ship go to a farther planet to get items instead of a closer one？"
Good question! In today's dev log, we will introduce the history and evolution of the interstellar logistics system. We'll roughly explain the iteration process of optimizing the interstellar logistics system experience.
[h2]Before the Update
[/h2]
In the previous interstellar logistics system, the transportation of an item from interstellar logistic station A to Station B was facilitated by a mechanism called Supply-Demand Pairing. Each supply-demand pair contained information about the supplier, the demander, and the item being transported.
Each interstellar logistic station managed all its supply-demand pairs internally. It needed to check with all other stations to see if its supply and demand could match with those of other stations, and then store all the matched supply-demand pairs.
All these supply-demand pairs formed a complex logistics network. For example, if there were 10 interstellar logistics stations in the universe demanding iron ore and 100 stations supplying iron ore, there would be 1000 supply-demand pair routes regarding iron ore.
After obtaining all the pairing information, logistic stations need to undergo periodic updates. As you all know, interstellar logistic stations are very convenient to use. You just take out items from your inventory, place them on the ground, set the items, and the goods start to be transported continuously.
Therefore, generally speaking, the number of interstellar logistic stations in the entire universe is not too small. Consequently, the pairing information within each interstellar logistic station grows exponentially.
To balance the entire supply-demand relationship (as well as optimize performance costs), each time a station judges the pairing information for picking up goods, it adopts a "fair share" strategy. This means traversing through all the pairings one by one and marking the last position after the traversal ends. The next traversal starts from this position to ensure that each pairing has a chance to be "loved".
Behind the vast transportation routes, thousands of stations and tens of thousands of supply-demand pairings perform logical computations every second.
Seeing this, you might already feel your brain getting a bit fuzzy, right? But the complexity doesn't stop here!
To avoid wasting transportation resources, if a pairing passes various logical checks (such as availability of goods, energy, transport ships, etc.), it also needs to consider whether it can conveniently deliver some additional items.
For example, when a vessel destined to pick up iron ore sets off, it needs to reevaluate if it can conveniently bring a small gift to the supplying station on its way. Similarly, when a vessel from the supplying station arrives at the destination and unloads the iron ore, it needs to check if it can conveniently pick up some other items to take back.
Our intentions were good, but things went awry during execution. Code is rigid. It doesn't care whether a pairing is closer or farther away. It simply follows the rules: if a pairing is detected and passes all logical checks in this frame, then that's the route to take!
This puts a strain on our engineers, especially when transport ships are relatively slow in the early stage. It's frustrating to watch a transport ship ignore a mining area just light-years away and instead head straight for a destination tens of light-years away, never looking back.
[h2]Beginning the Optimization[/h2]
When the CentreBrain requested optimization of the interstellar logistics system, the designers were quite unwilling, even resistant at the beginning. The previous logic was already complex enough, and further optimizing the experience would only lead to considering even more complex situations.
As the saying goes, the complexity of design doesn't decrease; it shifts. Now that we have such a complex situation causing experiential issues, the only solution is to devise even more complex rules to match these complex demands.
After continuous battles between designers and programmers, we decided to expand the "pairing" mechanism into multiple dimensions. We allowed engineers to set priorities across different dimensions, ensuring that engineers could adjust the priorities by manipulating pairings across various dimensions. This approach aimed to reduce the occurrences mentioned before and enhance engineers' sense of control over the entire cosmic factory.
We can do it!
The new design of supply-demand pairings consists of five dimensions:
① Point-to-point priority pairing between individual interstellar logistic stations.
② Setting up "routes" between planets for specific items, with priority pairings for stations along these routes.
③ Setting up "routes" between star systems for specific items, with priority pairings for stations along these routes.
④ Grouping interstellar logistic stations, with priority pairings for stations within the same group.
⑤ Regular pairings for items not assigned any specific priority.
Their priority decreases sequentially, meaning that interstellar logistic stations will first check if point-to-point pairing (①) is feasible for transportation. Then, they will consider dimensions ②, ③, and ④ in order. Finally, they will only address regular pairings (⑤) if necessary, based on the settings.
After setting up a planetary route between the third and fourth stars, titanium glass will be prioritized for transportation between these two planets.
Just as we were eagerly diving into development, we encountered a problem after a week of joyful coding — the situation was far more complex than we had imagined.
During testing, we found that in specific situations, logistic stations planned for point-to-point pairings(①) couldn't transport due to reasons like lack of warpers or energy. They could still transport (either delivering or fetching) when considering regular pairings (⑤).
After obtaining items from regular pairings, when it's time to check point-to-point pairings (①), it may be discovered that there are no items available. Alternatively, in cases where transportation is not feasible in ①, some demanders from lower-priority routes might opportunistically "steal items". Because they do not know why ① doesn't work, whether due to a lack of energy or a lack of warpers, the handling methods for these situations are different.
Moreover, the lack of energy needs to be categorized into situations where power is disconnected or not. Some situations require waiting for higher priority, while others cannot wait.
If these situations are not addressed, low-priority pairings will almost always override high-priority pairings, leading to the prioritization meaningless.
A simplified transportation model, where arrows represent transportation direction, and double arrows represent high-priority pairings.
So we adjusted the design of priorities. Priority no longer completely determines who goes first and who goes later. Moreover, in situations where high-priority pairings cannot be transported, it is not guaranteed that low-priority pairings will be served next; it depends on the specific circumstances. By studying the departure conditions of transport ships, we found that there are 9 situations to consider for each station.
Therefore, there are a total of 9x9=81 different situations to consider for both the supply and demand sides.
The designers and programmers felt "intimidated" looking at this table.
Bad news: every 1000 logistic stations will increase CPU overhead by approximately 1ms on average.
some fake codes
To implement these mechanisms, not only do we need to synchronize the timing of updates, but we also need to introduce a "locking" mechanism. When a pairing is transporting items between each other, they will apply a lock to each other. If a lower-priority pairing attempts to transport items and finds a high-priority lock on the logistic station, it will "back off".
Conversely, if a higher-priority pairing attempts to transport items and finds a low-priority lock, it will simply ignore the lock. Of course, this is just a simplified situation. More complex situations, such as when a higher-priority pairing cannot transport items, are not discussed here.
A Shipshape Panel
[h2]Thank you all[/h2]
After this update, the brand new interstellar logistics system is about to meet all the engineers.
We appreciate everyone's patience during this period without new features, and we are grateful for your continuous support and love for "Dyson Sphere Program".
As mentioned earlier, the complexity of design doesn't decrease; it shifts. However, we will always carefully consider engineers' needs and will not give up simply because of design or code complexity. In the days to come, we will continue to bring updates "Dyson Sphere Program".
Thank you all!