Welcome to our latest Inside Infinite, our monthly blog series that provides a closer look at the development of
Halo Infinite. This month, we’re diving into efforts supporting the upcoming multiplayer technical preview, often referred to as “flighting”, to share insights into our goals, process, and what Insiders should expect. We’ll also take a closer look at the content and experiences included in this inaugural tech preview which will focus on Bots and Weapon Drills – two brand new additions to Halo.
We have done quite a few Inside Infinite blogs already this year, and we highly encourage you to check them out if you’d like to learn more about our development process.
These articles are made to help bring you closer to our studio’s process, share our goals for the game, and present new aspects of
Halo Infinite in a unique way. Before we get into the core of this Inside Infinite, if you’d like a chance to participate in any Halo Infinite technical previews, you’ll need to have a verified
Halo Insider account.
Halo Insider
- Register for Halo Insider using your Gamertag
- Verify your email by clicking on the confirmation email sent to your Preferred email address in your Halo Insider profile
- Opt-in for communications by selecting “I would like information, tips, and offers about the Halo franchise.” – You’ll need to verify your email and check this box, otherwise we won’t be able to send you flighting emails
- Opt-in for flighting on the platforms of your choice (console and/or PC). To be considered for PC flighting, upload your DxDiag and connect your personal Steam account
Please note that
registering as a Halo Insider doesn’t guarantee access to technical previews or pre-release hands-on gameplay tests (“flights”). The number of Insiders being invited, and the criteria by which these Insiders are selected, will vary given the specific scope and goals for a flight. For the tech preview, we have concurrency targets tied to our overall goals that will inform how many total Insiders should be invited. Within that pool of players, our first priority will be ensuring we have a broad distribution of platforms (Xbox One, Xbox Series X|S, and different PC configurations) and then we will look at overall tenure in the program, prioritizing our longest standing Insiders, until we've allocated every invitation.
We’re excited by the outpouring of support and interest across the community, and the Halo Insider database has grown considerably, which is great news as we eventually look to stress our services and systems at an even greater scale. However, this also means that not every eligible Halo Insider is going to get an invitation – particularly early on as we are deliberately taking a “walk before we run” approach. Our goal is to offer additional technical preview opportunities ahead of launch with increases in scope for content and participants. So, if you don’t receive an invitation this time, we ask that you please hang in there – future opportunities are on the horizon.
With that out of the way, let’s dive into the depths of what it takes to create and more importantly, support and operate, a flight for a game like
Halo Infinite.
COME FLY AWAY WITH US
Before embarking on any game development journey, our teams spend time identifying the needs of our games and players, establishing measurable goals and milestones to ensure we’re meeting those needs, and planning how best to verify that we are, in fact, meeting those targets. Practically every aspect of game development is vetted and planned this thoroughly before it can become a reality.
Over the last few years, this same process was instrumental in the journey to update and improve
Halo: The Master Chief Collection and then bring it all to PC. This same flighting model and partnering with Halo Insiders has been a cornerstone of
MCC’s ongoing seasonal updates as well. It’s safe to say that without the support and partnership with Halo Insiders and a robust flighting program, all of the goodness that’s happened with
MCC wouldn’t have been possible.
Now, applying that process to
Halo Infinite, we had goals to
robustly test our game and core services at a large scale and allow players to provide feedback on the game before launch. Next, we focused in on
outlining how to measure progress toward these goals. A hypothetical example of would be, “we want to have a matchmaking success rate of 100%.” That’s a great goal to have, but then the real work begins building systems that can reliably
measure whether we are hitting these goals or not. In the matchmaking success rate example, this means adding code to the game client or our online services that can monitor and report when matches are made successfully and, more importantly, when they fail.
Since our larger goals were dependent on what happened when players went hands-on with the game, we knew involving Halo Insiders for a technical preview would be the best call. Hosting a large scale flight with Halo Insiders will give our team plenty of gameplay data to determine whether or not our online systems (matchmaking, challenges, etc.) are working as intended. It also gives us an opportunity to gather – and measure – feedback from all kinds of players, from all over the world.
Now that we’ve talked about the ‘why’ behind hosting technical previews with Halo Insiders, let’s take a closer look at what a “technical preview” really means and the specific goals the team is working towards.
[h2]TECHNICAL PREVIEWS, FLIGHTS, AND BETAS[/h2]
What’s in a name? Quite a lot, as it turns out. Different games across the industry, and even Halo games over the years, have had all sorts of pre-launch hands-on releases and tests that carry various official labels. Terms like “alpha” and “beta” often convey a certain expectation around polish and stability, breadth, marketing implications, and more. We’re further along than what would be truly considered an “alpha” but also not really at a level of overall completeness some expect in a “beta.”
We are specifically calling this a “technical preview” because we feel it best represents the goals we have, the build itself, and the experience that Insiders will have. As you’ll read below, our key driver and goal is truly technical in nature – we are looking to push our systems and services at a larger scale than we’ve been able to thus far as an important step towards ensuring we are as ready as possible when the flood gates fully open at launch this holiday. Feedback and other insights are certainly valuable, but first and foremost we are testing and stressing the technical side of
Halo Infinite.
We also want to be very up front and transparent about the fact that we will certainly have some bumps and rough edges in this build. Our teams have been working hard to ensure that the build and experience is such that it can deliver on our goals, but this is very much still work-in-progress, and we expect some hiccups of varying degrees over the course of the technical preview. For further context, this technical preview is a couple months behind where the development team and ‘main’ game build currently is given the process and time it takes to go through all the gates and steps before releasing to the public.
All of our key known issues will be listed so participants will have a better understanding of what to expect and what to look out for. Of course, gathering support tickets on brand new issues is also immensely helpful at this scale – despite our team’s best efforts, nothing can truly substitute for true real-world large-scale testing, and we’re sure to find some new areas of opportunity over the course of the preview.
And lastly, we’ve already mentioned this a few times in the blog, but the word “flighting” is fairly new in a lot of circles so when you hear us mention a “flight” – we are simply referring to a work-in-progress, hands-on release on the road to launch. Flighting is the process by which pre-release builds are put out into the wild to pursue specific goals, review data and outcomes, iterate, and repeat – all with the overarching goal of delivering the best shipping experience come launch time.
With that nomenclature and expectation-setting out of the way, let’s take a closer look at the key goals the 343 Industries team has for the
Halo Infinite technical preview.
[h2]TEST THE GAME AND OUR CORE SERVICES AT SCALE[/h2]
We want to make sure the game and all of its supporting services are operating as expected, especially under the larger-scale pressure of a technical preview. By flighting the game to Halo Insiders, we’re going to see players uncover quirks in our online services, find whacky new bugs in the game and menus, and so much more. The beautiful part of this process is that it’s all intentional – discovering these issues, small or major, months ahead of launch is the entire point of flighting. We want to find these now so that we can address as many of them as we can ahead of launch.
Let’s take a look at some of the areas we’ll be monitoring during the upcoming technical preview.
[h3]GAME & SERVICES PERFORMANCE FOCUS AREAS[/h3]
- Stability: How stable is the game across various hardware specs and platforms? Is the game freezing or crashing at a certain time or on specific hardware? Getting a large-scale sampling of real-world hardware profiles is something we can only really accomplish via a large flight of this nature and as we launch on PC for the first time, this is a critical focus for the team.
- Online Services: Can all our interconnected online services function well at an immense scale? This is our largest bucket since it includes so many intertwined services that need to function in perfect harmony. Matchmaking, server scaling, playlists serving the proper content, challenges, stats (Post Game Carnage Report), Battle Pass progression & Store functionality, armor customization, partying up with friends, etc.
- Halo Insider/Waypoint vNext: Can our Halo Insider communication systems and flighting tools function at scale? How do the Waypoint vNext app (iOS & Android) and web experiences feel and function as extensions of the game?
MEASURING GAME PERFORMANCE
Our team has already put in the hard work developing the tools to measure and collect information about nearly every scenario imaginable, such as game crashes or matchmaking failures, so technical previews will give us a unique opportunity to compare our internal data to what we see show up in the wild. Measuring the game’s stability and performance, especially at scale, is an integral part of a game’s journey to launch.
Now, let’s hear from members of the Halo Infinite team to see what they’re monitoring and how they’re measuring it during our upcoming technical previews.
What are you/your team monitoring during flights?
- Sam Hanshaw – Producer, Live Team: We’re always looking at the rates of matchmaking success and the availability of our servers, after all it’s hard for people to playtest the game if they’re unable to play matches. We also keep an eye on how many matches are getting played to completion, and we receive data about how many people crash while playing.
- Brian Dunn – Multiplayer Test Lead & Brian Hughes – Live Test Lead: The test team monitors a large number of specific areas over the course of a flight. First, we are watching for overall crashes and stability. We analyze this data on a per-platform basis and identify specific things like most frequent crashes and differentiate between known and new crash instances. We use a metrics such as “MTTF” (Mean Time to Fail) or “crashes per 1,000 hours of gameplay” as a way of quantifying overall stability.
Next, we are also closely monitoring the overall quality of matchmaking which includes reliability, load times, and skill matching. In addition, the performance of our dedicated servers, performance of the game itself (particularly across different PC configurations) and our overall netcode quality are all under scrutiny. And with Halo Infinite's greater emphasis on player customization, we’re going to be paying close attention to progression, challenges, and which customization items people are unlocking and using.
Lastly, we partner with the Halo Support team to review tickets and bug reports to cross-reference against our telemetry and to help understand real-world player impact and scope of an issue. We also lurk in the Halo Waypoint forums, Reddit, Twitch, and social to keep an eye out for anecdotal reports.
- Nate Jones – Engineering Lead, Services Lifecycle Team: If everything’s going well, I’m able to play some matches during a flight and I’m not glued to logs/monitoring. I keep the lobby service dashboards up to the side during the flight (watching backend service performance health, dedicated server usage, number of players connected, matchmaking ticket success rates, etc), and I’ll occasionally get pulled into a mid-flight investigation and have to dig into the logs. Most of my (and my team’s) flight monitoring takes place after the flight ends. Our normal post-flight plan is to thoroughly look over the lobby and skill service logs for errors, warnings, or ‘weird stuff’; that’s on top of any deeper dives into specific experiments we were running during a flight.
- Jeff Guy – Producer, PC Team: I’m going to be monitoring Brian Dunn for cheating because he wipes the floor with me every time I play against him in a match. I’m only half joking. :P
We've built a TON of different player-controlled settings into Halo Infinite, as well as support for a vast array of player hardware and peripherals. The PC team will be monitoring how the game holds up across that matrix of player choices. We’ll be looking for things like drops in framerate and crashes that happen with specific hardware configurations. We’ll be combining that information with their graphics settings and peripherals – how many people are rocking ultrawide setups, who is pushing 4K or 8K resolution, who is playing with unlocked framerate versus locked at 144, what custom key bindings are players choosing, etc? Ensuring our PC players can play Infinite the way they want is everything to us, and we’ll be closely watching for any problems when they do.
How do you/your team measure that information?
- Sam Hanshaw: We have a general release health dashboard that displays information on matchmaking errors, and any crashes that are happening. There is also a wealth of information in the server logs that the team can dig into during investigations to figure out what broke and when. We also have a great support team who are capturing bug reports and support ticket submissions, these are what we’ll dive into after each flight is completed to get a picture of the experience through player observations of bugs. The information we get from those tickets is valuable for helping us track down the issue, and also for knowing how many individual people an issue is bothering, and how often it’s bothering each of them.
- Brian Dunn & Brian Hughes: We have a robust suite of tools and systems that feeds critical information into our team for real time and reactive monitoring. Any time a player crashes, a system called “Watson” uploads a detailed crash report which feeds into our internal crash reporting website, “Ticket Track.” Like Nate and the Services Team, we also use Kusto which helps us understand the data we’re receiving and converts it into something useful for reporting purposes. Speaking of reports, we use Power BI reports for any of the data we need to visualize and then all of this feeds into our Azure DevOps database for bug tracking. Our team uses data from all of these sources to get an understanding of the quality of the flight, and ensure we're making the right improvements for current and future flights.
- Nate Jones: We use Kusto (Azure Data Explorer) for most of our ad-hoc data analysis; I spent a lot of time writing queries that join service, dedicated server, and client telemetry events together to figure stuff out. We have a bunch of pre-built graphs and dashboards using an internal Microsoft system called Geneva. We’ve also got a few internal tools (bespoke websites). One’s called “lobby logs”; it’s like a mini search engine for the Halo lobby service that can trace a player’s parties/matches and correlate their sessions with the other players/parties involved in the matchmaking process or eventual game session.
- Jeff Guy: We have a lot of tools and telemetry already set up to capture a lot of this data, even at scale. When someone crashes, for example, the game will automatically upload a crash report to us. Our team can then look at the details of each specific crash or use that information to spot a pattern that could indicate a larger problem. In terms of players’ settings, we’ll get telemetry data when players save their key binds and video settings, since a good portion of that information is saved in our online services. We can even check to see if a player’s framerate drops during a session. Gathering the full picture of all the different settings and variables enables us to quickly narrow down our investigations and fix issues fast.
What do you/your team consider a success?
- Sam Hanshaw: We want players to be able to consistently play matches of Halo Infinite. The more matches they can play, the more issues they can find for us to deal with. Every new issue players are able to find that we were not is a success for me. Anything that gets found during flighting is something that’s found before the game launches, resulting in a better game for everyone when we release.
I know I’ve talked about problems a lot, but overall, we also consider a flight a success when we see people having fun playing the game. In the studio we’ll be in the flight with you, and while we’ve got a ton of information we need to gather to make this game the best it can be at launch, we’re also looking forward to playing alongside you and opening a dialogue about our road to launch. To everyone who is going to participate in our upcoming flights I want to say thank you for helping us make this game better!
- Brian Dunn and Brian Hughes: Success for us is when we're able to meet or exceed our internal targets, while not having too many big surprises. You always learn something new when so many people play your game for the first time, but hopefully there's nothing completely unexpected. We’ll be looking specifically at our matchmaking quality real world results vs. our targets as well as our crash counts and rates actuals vs. what we anticipated. We also hope to see player feedback and data coming in that aligns with our own internal expectations for overall quality at this stage of the project.
- Nate Jones: Selfishly, one hallmark of success is when nobody needed to activate my team’s on-call process and phone someone for mid-flight engineering support 😊.
At a high level, success is getting usable data from the flight. Good news is great, but bad news is still valuable. We’re always looking at general health (CPU/mem/bandwidth, matchmaking speed and quality, etc) and we’ll have general quality/performance targets, but any given flight also has specific experiments we’re running. For example, during an internal flight in April, we did some explicit tests around how our pool of dedicated servers would fallback/failover to secondary pool(s) if we run low on machines. That April test was technically a failure (we didn’t see the expected players using the secondary server pool), but we narrowed the issue down to a bug in an external team’s system, and they were able to find/fix the problem from our flight data.
- Jeff Guy: Creating a first-class PC game is all about embracing player identity and all the awesome diversity that comes with it. PC players build their gaming rigs to reflect the way they want to play, and we are making sure Halo Infinite honors that. The challenge of course is that supporting such a diverse array of hardware and software creates a lot of ways for things to break.
Our team considers flights a success if we're able to find issues that we might not have seen if we didn’t put the game in the hands of our players early. This is critical to ensuring every PC player within our min-spec has an awesome experience on day one. Also, we’d love to see which custom key bindings players use...or better yet, see players choose to use the default key bindings because that would mean we chose them well. :)
We appreciate you taking the time to share your insights and expertise with our community today. It’s reassuring to know that the issues they may encounter during flighting are usually seen by the team, and there’s rarely a need to wonder, “Is 343 seeing this?!” since you have built the tools to do exactly that. We'll let you get back to preparing for the technical preview now, thank you again!
PLAYER FEEDBACK
Gathering input from our players during a technical preview is critical to our success at launch and beyond. The sooner we receive feedback, the sooner we can plan and action on that feedback. On that note, we want to clarify that with this technical preview we are entering the next step in what’s been a multi-year process and journey where the team has been regularly getting critical feedback via user research and private flights. Naturally, the scale of the technical preview is greater than anything we’ve done to date – and that brings unique value and opportunities to capture even more perspectives – but just know that community feedback has been at the core of development since the very beginning. And, it will continue to remain a critical pillar going forward beyond launch.
Now, let’s take a look at what areas of the game we’ll specifically be focusing on for feedback during the technical preview.
[H2]FEEDBACK FOCUS AREAS[/h2]
- Core gameplay: We want to know how you feel about the core combat experience. How do you feel about player movement, weapon balance, equipment, etc.? How fun is it to engage in a battle in Halo Infinite?
- Maps & modes: How did you feel about the design and flow of the maps? Are the maps and mode both fun and fair?
- Academy’s Weapon Drills: Did Weapon Drills give you a good understanding of how a weapon functions? How fun were the various Weapon Drills?
- Menu & Battle Pass UI: How intuitive was the menu navigation? Were you able to understand your progression through the Battle Pass? Were you able to find and equip customization items easily?
[h2]MEASURING FEEDBACK[/h2]
While it is critical, player feedback can be a bit of a nebulous cloud of forum posts, tweets, and YouTube videos - which can make it hard to measure in some cases. Luckily, we have a two-pronged approach to gather this information, measure it, and then share it with the team. The two prongs in this case are the Xbox User Research team and the 343 Community Team.
The Xbox UR team gathers measurable, quantifiable data through surveys that will be sent out at the end of a flight. These surveys ask detailed questions that drive deep into the core of a player’s experience in a flight – and turn that experience into measurable data. Since we have survey data from the entire history of Halo, they help us compare our current scores to the best scores from across all games. We can also compare scores from different times (or flights) of the game to see how specific aspects are progressing over time. They put together some of the most helpful reports we can read, and it’s all coming straight from what you – the players – are saying about the game.
The Community Team dives into the amorphous realm of online interactions mentioned above. We set up focused feedback threads here on the Waypoint forums, listen to players on Twitter, check the Halo subreddit multiple times a day, watch videos and streams from Halo players, etc. We take all those posts and videos, and distill their messages into positive, neutral, and negative themes. When doing so, it isn’t just about volume though – anyone can send a lot of tweets. Our team focuses on the severity (how much does it affect an individual) and scope (how many players), when reporting on these themes.
More often than not, even though we use different ways to collect player feedback, the Xbox User Research and 343 Community teams see similar themes emerge. These two inputs help the team compare
Halo Infinite’s current state with its own previous scores, previous Halo titles, as well as the development team and player expectations. Using these comparisons, we can recognize which areas need improvement, which areas are performing as expected, and which are exceeding targets. After reviewing this feedback, we identify opportunities and prioritize the areas we want to go after.
[h2]TRANSPARENCY[/h2]
It’s important to note that while we are going to be listening closely, there will be some feedback that we can act on before launch, some feedback we’ll have to fit into the game’s roadmap, and additional feedback we won’t or cannot take action on. That said, we will communicate what will and won’t be changing in the game as a result of the technical preview – and why.
As part of this commitment to transparency, the team will meet to go over hot topics and common pieces of player feedback that emerged during the flights, which we call “themes.” After reviewing these themes, the team will spend time investigating, figuring out if they agree and want to act on it, and when in the schedule we can take action on them. Again, there will be things we can’t act on before launch this holiday – and that’s okay – we plan on supporting Halo Infinite for years to come and this feedback will all be captured to help inform future priorities.
Once we’ve done the due diligence on our side, we then work on putting together an “outcomes” report that outlines our plans for each and every one of those key feedback themes. From there, we’ll work to surface that information to you, our players, likely through a post here on Halo Waypoint.
[h2]BUGS & ISSUES[/h2]
When it comes to bugs and issues specifically, we ask Halo Insiders to utilize the Halo Support (
aka.ms/HaloSupport) site to file a ticket. When the technical preview goes live, everyone who is invited will receive an email and Waypoint message that includes some important links to known issue documentation along with where and how to file a support ticket. To help keep everything tidy, a special section of the Halo Support site is created exclusively for Insiders who are invited to a current active flight. This helps us ensure that tickets are only coming in from players who are actually participating in the flight and it keeps flight tickets fully separate from retail tickets (particularly in the case of MCC). As always, we ask that you read the known issues list first, so you have context for issues the team is already aware of and thus don’t require a support ticket. Anything a player encounters that’s not on that list is incredibly valuable to the team and we thank you in advance for taking the time to submit these tickets!
UPCOMING TECHNICAL PREVIEW
Now, with the foundation of flighting set, let’s talk about the content and goals of our first upcoming technical preview, which will focus on Bots and a slice of the new Academy experience.
This technical preview is intentionally focused on these specific areas to help us gather some scale data as we bring these new experiences to Halo for the first time. As we look further ahead, we plan for future flights to expand into more content including traditional PVP Arena and Big Team Battle.
With this and our higher-level feedback goals noted up above, we land on the following specific focal points. With this in mind, the focus areas for our technical preview are:
- Arena gameplay vs. Bots
- Arena Maps
- Academy’s Weapon Drills
- Menu & Battle Pass UI
- New Waypoint experiences on web, iOS, and Android
To gather specific feedback on these areas, our first flight will contain the following content and experiences.
[h2]BOT ARENA[/h2]
Halo Infinite is the franchise’s first foray into the world of multiplayer Bots. To provide a totally new multiplayer experience for players and to gather feedback on Bot behavior and online performance, we’ll be debuting the Bot Arena playlist, which puts 4 players against 4 Bots on Arena maps.
For this technical preview, the Bot Arena experience will feature Slayer across three maps in this build:
- Bot Slayer on Bazaar, Recharge, and Live Fire
To keep players on their toes and keep the experience engaging, we’re planning for a daily content offering that will evolve over the course of the flight based on the overall engagement of participants. We’re going to intentionally start things off slow to help everyone get acclimated but as the community gets up to speed and slays copious amount of Bots our goal is to crank things up to provide greater challenge and variety.
Bot Arena is a great place to familiarize yourself with
Halo Infinite’s gameplay and maps in a stress-free environment. The Bots still have their quirks, which is why we’re flighting them, but they certainly aren’t pushovers. Be prepared.
For more info, including information on Academy Drills, the Waypoint App, words from
Joseph Staten himself, and more,
be sure to check out the rest on Halo Waypoint!
An interior look of Bazaar. [For the full-res image, 
Introducing Fret, a Personalized AI. [For the full-res image, 
