Victoria 3 News

[h2]Introduction[/h2]

I lied!

We’re not talking about Audio this week, we’re talking about Quality Assurance, otherwise just known as QA! Excuse the dust as we teardown and set up this new development diary a little earlier than planned. We will eventually have an Audio Dev Diary (looks hopefully at Community) but for now you’ve all locked in here with me again about a subject I know all too well: how it is to QA Victoria 3.

Though this is honestly a great introduction to what it's like to QA Victoria 3, or how it is to be a QA in general. As we are at the end of the pipeline we must be the most adaptable members of the team. Broken builds, delays, reworks, and merge errors - these are an everyday occurrence in our profession. Even when things go perfectly, QA is still a race against time or more specifically a race of timeboxing: what is the most important thing to test now and what can be put aside to when there is either time or it is more complete - do we suspect another Market Rework is coming or are we at the final implementation at last?

A regular QA posting, usually among ourselves.

Not everything can get tested, it's a sad fact. Paradox games are unique in their complexity of interlocking mechanics and systems that to test the full functionality of the game and all of its iterations for every change is impossible. An infinite number of QA with an infinite budget and time could still not deliver a near perfect quality product. Why is that? Well because in that scenario the bottleneck of quality would not be the QA but the developers on the other side of the bug fixes - it's their bandwidth that would then be the limiting factor. The Job of QA is to test and test appropriately balancing the time for technical feedback and gameplay balance based on the development timeline of the game.

And before we get too far, the comment that I expect to get the most to this diary:
“Why don’t you use [insert specific development/testing process], it will solve all your problems!”

I wish this were true, but there is no silver bullet for development.

Each model of development and testing mitigates risk in a separate way, but there is still inherently some risk especially because we are talking about the potential for human error: missing connections or building on top of a system that was not originally written with such a perspective multiple iterations ago.

The most lock-tight development model that does not allow for any “bugs” also does not allow for any creativity on the part of the developers. Video games may be a product to be sold but they are also inherently an art, which makes QA’ing and solving their problems a much more subjective experience at best. The trick is to find a system that creates as many safeguards as possible without fully boxing in the creativity of the team. I digress and will stop there before I get into a longer wided sidenote. Throughout this diary I will go into more detail about what I mean.

[h2]The People, The Process, The Timeline, and Going Forward[/h2]

As we’ve moved through the more public side of the development process, my role in QA has shifted while the meme of me as a QA Lead has remained the same. I’ve gotten a lot of questions about “what exactly a QA Lead does and how my role as Manager/Director was different” so I thought I would take some time to flesh this out for you all with at least a basic explanation.

The People - QA Team Breakdown:

QA Director: Represents the QA team’s interests on the studio leadership level, ensuring that all plans of release and development are taking into consideration the bandwidth and requirements of their team.

QA Manager: Professionally manages the QA Team, less involved in the day to day and more involved in the growth and development of the QA Team as individuals and overseeing staffing.

QA Lead: Represents the QA team’s interests on the project leadership level, manages day to day prioritization and coordination of testers.

QA Tester (Embedded): Employee of Paradox Development Studio, who lead the charge in testing development, verifying fixes, and giving feedback.

QA Tester (Outsourced): Employee of our Outsourcing Partners (such as QLOC) who works with the internal team and assists in testing development, verifying fixes, and giving feedback.

Community/Beta Tester: Paradox Community Member who under NDA gets access to an in-progress build to give feedback and partial bug reports on - which the QA will later collate and verify.

Overall QA are an interesting bunch of people. It takes a very dedicated individual to look at an unfinished project and happily get to work, to not only document the flaws but to peer behind the curtain and see the intent of the finished product and give feedback on that. The QA, as an individual, is able to look a developer in the eye, tell them they love a feature and then send them 20+ jiras about all the problems with it with no cognitive dissonance. Sometimes we will throw in a few feedback jiras about how we think it would be cool to tie it into other features as well.

Checking if the issues are actually fixed is always an adventure - you never know which new issues you're gonna find

Our need to be critical and take the player perspective also leads to us being very blunt and forward when asked our opinions. In our profession, sugar coating concerns only lead to problems down the line. The trick that I will share on the forums is this - it's possible to be blunt and not a jerk. Browbeating a developer as a QA may get the problem solved but it comes at the cost of the working relationship. I have been many types of QA in my career and if I may be so bold to reflect upon it - it is the QA that is blunt yet respectful that gets more achieved in influencing the game then the one who screams the loudest. As developers are people too, they have feelings and can react emotionally especially if you act emotionally (especially angry) towards them. I ask that you please remember this as we move towards post launch and live support, but I will get into that more later in this dev diary.

We were once told that we should go about things in a more positive perspective. That request did not stick for long.

I’ve joked on stream that “I saw the QA team laughing during multiplayer testing and that’s either a sign that everythings working or everything is broken.” Our profession comes with a strange sense of humor, or maybe it's just the type of people who enjoy our work - after 8 years I’m still not sure.

[h2]The Process[/h2]

I’m going to talk more about the general process of QA and try to avoid going into too much detail here, otherwise it is a rabbit hole I will never escape. What I will say is this, I am talking high level processes of the QA department as a whole in relation to the project, QA’s interfacing with each other department: Code, Design, Narrative Content, VFX, Audio, UI/UX, Environment Art, Tech Art, 2D Art and the various subcategories I’ve forgotten - each of these have a separate process work a bit differently. This is because they are all developed differently and implemented at different times of the game and so the standard QA has for one team is different from another depending upon the state of development.

QA has two types of general process testing, Milestone and Feature.

Milestone testing is the one most people outside of the industry understand as it translates to the words we use - Proof of Concept, Alpha, Beta, Release Candidate, etc. These are all milestones that we use in the development process. The intention is that there is a significant difference between them, the status of its features and even more importantly its gameplay loops and engagement. Milestone encompasses the game as a whole and is generally used in the statements such as “its as you would expect for alpha” etc.

Even while testing the features for milestones. QA, just like the rest of the team finds a way to add a little humor into the documentation.

Feature testing is a bit different, here at Paradox we usually refer to it as “development support testing” it's where QA is doing more piecemeal testing of individual implementation of the various pieces of the game. This is where we are ensuring that things merged correctly, that coders did not accidentally overwrite each other and we are caring more that things are working from the technical perspective then the full gameplay perspective.

When a game is in its early stages of development to its alpha milestones - feature testing supersedes milestone testing. That’s not to say we do not do alpha milestones but a large part of our testing is in the technical ends of the game as opposed to “playing for balance purposes.” The reason behind this is that while the game is still actively being developed and is not yet towards the milestone where features are less likely to change, testing balance is inefficient effort. QA has a limited amount of time and a never ending series of tasks to tackle, so we have to be as smart as we can.

Early on the fact the game is less polished has to be accepted by the QA team. We find unique ways to cope with this fact.

In case you were wondering, this is what that smile looks like today.

As we get further along our milestones, towards beta and release candidates - the inverse is true: milestone testing supersedes feature testing as the main means of gathering data. In beta QA may still dive into features and test them from a technical perspective depending upon the number of changes made by a developer but more often than not they will focus on how these features interact with each other and do they make for meaningful and fun gameplay. Our difficulty in doing this? We need to figure out how to answer this question with some form of certainty as quickly as possible.

As the game’s development progresses so too does QA’s standard processes on what is worth bugging and what is an acceptable fyi issue. Technical concerns have priority in early stages of milestone development while in release candidate papercuts with poor UI or incorrect information given to the player are treated as the same severity as broken functionality outright. Because it doesn’t matter if it technically works if the player cannot understand it.

[h2]The Timeline Going Forward (and Farewell)[/h2]
A QA’s role adapts as the project progresses. In the beginnings of the project we’re very technically minded and working on ensuring the structure of the game is there. As development progresses we focus more on the features and how they feel and the overarching scope of the game. As we get further along the milestones QA becomes the representative experts of their game. Designers can tell you how it's supposed to work, QA tells you how it does currently and because of this knowledge we work regularly with community, beta testers and influencers in teaching them how things work early on and getting their feedback.

Community is very much our comrades in arms in this endeavor, anything reported to Community Team on our forums or discord is brought to QA’s attention to ensure we’ve got it Jira’d and the rest of the team is aware.

Soon release will be upon us, an exciting time for us all. You all will finally get a hold of the game and the development team will sit here anxiously waiting to hear your feedback and see the shenanigans you all get up to. The QA team itself is going to sit here wondering what last minute changes we made broke what and how that slipped through the cracks of our testing.

At release and going forward we will have a bug forum where any issues found by you all that you think we do not know about, you can let us know. And even if we do know about it, you can help judge priority of fixing but more information will come in regards to that from the QA Team Lead around release so keep your eyes peeled.

And well, that’s a summary of high level QA processes, instead of talking about the nitty gritty of testing X features or how many jira’s we’ve written I hope you enjoy the peak behind the curtain to our sense of humor splattered throughout the dev diary.

I guess this dev diary also marks the end of my responsibilities as QA on the team. I now go off to the other side of the merge requests… ready to take on a different kind of responsibility. Be kind to my QA Team in the coming weeks. You have no idea how much effort they have put into this game over its development - especially in the past year. I certainly do, and I’m very proud of them. The project is in good hands.

Hopefully next week we’ll have our Audio Development Diary with Franco, sorry I’ve got no classy segway this time.

Victorians! Learn about the game in our series of tutorial videos! Today we look into resources: [previewyoutube]https://youtu.be/7sZNeu-ALWA[/previewyoutube]

Why not sit back, relax and let Paul talk about the economy of Victoria 3:
[previewyoutube][/previewyoutube]

Prepare for the release of Victoria 3 by dreaming of a better tomorrow while listening to the game's Official Soundtrack. Let the songs of an era of transformation and revolution be the background music of your everyday life, and imagine yourself the builder of a mighty mercantile empire. The Victoria 3 Official Soundtrack will be available on most music platforms.

Find a link to your favorite in the list below.:
Spotify Youtube Music Deezer Anghami Amazon 7 digital Apple Music

Hello all, today we are going to talk about some of the data visualization in Victoria 3, how we on the team have iterated on it since our UX dev diary and what we think through when talking about such iterations.

Aron is very busy doing what last bits of polish can be done before we lock down the game for release certification, so I have been asked to write this dev diary for you in their stead. I’m not officially a UX Designer but I do have my qualifications in user experience as QA and I also am quite opinionated about pie charts and other forms of data visualization as many of you in the community are no doubt aware of. I regularly get to be Aron and Henrik’s rubber duck as we talk through solutions to problems on the UX side and it's part of my job that I thoroughly enjoy.

(In fact the first draft of this diary was a full thesis on why pie charts are bad, but I was encouraged to tone down the rhetoric a little, so I will conclude they are aesthetically pretty but are horrible for conveying data.)

Just how angry is Paul? Even with the visualization you would still be guessing…

Now most of the time when we talk about Data Visualization in games, especially Paradox Games, the first thing that comes to mind is either line graphs or charts of some kind - giving a visual representation of data instead of just a pure numerical representation on the screen. This is not wrong, but data visualization is not solely about making some form of visual representation of abstract data. It is also about increasing the inherent cognitive understanding of the player by using known trends and patterns that are well established.

An easy example is the capacities, they are color-coded. When things are positive: it's green, when it's bad: it's red. We have a human tendency to impact meaning to color and thus it's a general rule of UX to never use Green and Red unless you are representing Good and Bad.

A quick glance at the capacities bar tells you all you need to know for the moment by color alone, and when you get its tooltip it continues such color coded summarization:

Have any of you who’ve seen streams or images noticed that this dynamic changes in regards to income? When your nation is in a deficit the balance can show either as white (neutral) or red (bad) in the capacity summary and there is an inherent reason for that?

We do not show a negative income balance as red until it reflects an unhealthy economy, but what do we mean by this? Nations run deficits all the time, but not all deficits are bad, especially those which are investments into the country such as construction. Construction is classified as a temporary national expenditure as opposed to a fixed one, meaning that we calculate your fixed revenue vs expenditure to be positive once the construction is finished, and we keep the income balance showing a neutral tone because of it.

This is to reflect that your money is going down, but the fundamentals are fine. What this allows you to notice is when this fundamental changes and turns red, signaling a larger issue of your economic fundamentals being out of balance that could cause future problems.

We may be losing money now but the fundamentals of the economy are okay if we ever stop building new factories…

This is a lot of what we have been trying to iterate on in Victoria 3 at this point in its development: we don’t want to just show you the static image of what’s going on just that moment but we want you to know the dynamic trends of where the data is going.

What do I mean by this? Well let me give you an example: see our buildings list (to make it clearer about what I am talking about I have gone with the minimized mode - didn’t know there was a minimized mode? Now you do!)

Behold the glorious minimized building menu and color-coded gold reserve bars.
(Bars are color coded on regular view as well)

Before cash reserves were only golden, and you would have to physically watch the bar tick along and hopefully notice the trend of your reserves. The data was still informative but you were only able to easily glance if your business had cash reserves (meaning it could afford a price disruption or could help supplement the investment pool). Now we use the historical data we would normally use to fill out a line graph to color the bar and add trend markers to show if the cash reserves is going up or down, coloring it green or red respectively.

Now at a quick glance you not only see how your economy is doing, but where it is going and you would be amazed at how significant such a small change can be. This is combined with our usage of red, white, and green to symbolize whether productivity is good or bad - to give you further depth of information: sugar industries in this picture are not failing but soon will be if you do not do something about it. Meanwhile the arms industry has stalled outright.

A lot of these data visualization changes, you might never see or notice, because when they work well their understanding becomes second nature, but here’s another one I want to call out: we recoloured the market summary tab:

Behold the new Market Panel, with corrected colors and balance bars to show the magnitude of the effect.

The older Market Panel, serviceable but flawed with its data presentation.

Okay we changed like 3 things, I hear you say - but why is that so damn important? I’m glad you asked that community strawman here’s why.

We removed the Red/Green visualization of the balance and by doing that we helped make our understanding of the market system clearer to the player. Remember, this is Victoria 3, imbalances are not inherently bad. Sometimes maintaining a shortage of a good can be done intentionally to prop up an industry or ensure the wealth of specific pops. (I went into the logic of why for this in my talk at PDXCON and if we are lucky at some point in the future I will get a copy of that up on the forums as well). We found that when we colored the items green/red we were inherently having players react in ways that they themselves found was not always good or what they wanted. In this case, this mostly meant players reacting to red numbers being “bad” and trying to make the number go green.

We also added balance bars, to help show you the magnitude of what those numbers mean in the scope of its total buy/sell volume and price scale. Can you tell from the old version above what the relative difference to your economy the liquor and sugar shortages are?). By giving more perspective on these offsets from equilibrium we help you better understand the cause and effect of your actions or most importantly; opportunity cost. If merely looking at high prices you might find yourself focusing on furniture or paper, but technically getting food prices lower is shown to have a larger impact on your economy, which is illustrated by the bars below the Balance values. The more blue or gold that bar is, the higher impact that Good’s imbalance has on your market. This impact may differ greatly per Good if it is good or bad, but that’s up to you to determine.

That’s just a few examples in the game of where we are iterating on data visualization. Mostly because these are the one’s I can easily remember and chat about. We are by no means done, Henrik, Aron and myself will continue in the trenches experimenting and iterating on such workflows throughout the game. We look forward to reading your feedback on release and use that to help us prioritize our backlog of ideas.

If you thought this dev diary was a sight to behold, just wait till you hear about the next one, which is the Audio of Victoria 3 which will be written by Franco Freda, our Head of Audio.