Rhythm Quest News

Despite what it might seem like, I've actually been working on quite a lot of different things for Rhythm Quest recently! Let's get right into it...

[h2]Game/Music Speed Mod[/h2]

You can now change the speed of the music to make the game more or less difficult:

[previewyoutube][/previewyoutube]

Some of you might be wondering why this took so long to come to the game, considering how this has actually been a debug-only feature for a while. Changing the speed/pitch of the music isn't actually very hard, the problem is getting all of the music synchronization (and respawn, and pause) logic to work properly in tandem with it.

I'm happy to report that this has been implemented properly now! It's definitely not as simple as it might seem...every time you change the setting in the pause menu, what I'd =like= to do is to immediately increase the speed of the currently-playing pause music loop, then adjust the audio timing variables accordingly. But there's no way to do that in a way that respect the audio synchronization perfectly, since the audio timings are all running separately.

Instead I have another copy of the pause music which is set to play at the new pitch/speed. Instead of trying to play that one immediately, I schedule it to start playing 0.2 seconds from now (scheduling sounds in advance is the only way to ensure that they trigger at a precise time on the audio timeline). Unfortunately, I can't schedule a "stop" command in the same way for the first version, so instead I simply wait for roughly 0.2 seconds and then do a flip where I set the volume of the first loop to 0 and the second one to 1 at the same time. Doing all of this lets me keep the beat synchronized with all of my timers correctly.

Of course, since the whole process does take a little bit of time (0.2 seconds), I also needed to implement a simple wait, in case you trigger the button twice in quick succession -- the second music speed change doesn't occur until the first one is finished.

Anyways, this will be available in the upcoming 0.28.0 patch, so players can either use lower speeds to practice / get through difficult sections of the game, OR use higher speeds to give themselves an extra challenge.

[h2]Timing Windows[/h2]

I also have a brand new setting for modifying the lenience of the timing windows for obstacles:



This affects the collider sizes of the various obstacles in the game. Here's how that looks for a basic flying enemy:



Originally I had a crazy half-baked idea in my head that to implement "extra leniency" I would actually buffer inputs for you and delay them automatically to "auto-correct" your early inputs...and for late inputs, I would just "freeze" the character in place for a split second to give you extra time to hit the correct button. I realized, though, that this would make empty jumps and attacks (the ones you do for no reason) feel really sluggish and awkward. I could try and do some tricks to correct for it, but in the end I figured that modifying the sizing of the colliders was simpler and just as effective, while maintaining the tie between your input and the game reacting immediately.

One cool thing is that the setting works for jumps too, even though you wouldn't think they have a "collider" to modify. This is because every jump (even if it's just a normal one over a pit of spikes) has an invisible trigger object (this is what spawns the blue "spinning square" visual effect). As long as you jump while you're inside of that trigger zone, my jump logic will autocorrect the length of your jump to make you land in the appropriate spot. I also already implemented "coyote time" jumps that will allow you to jump even after running off a cliff (as long as you're still within the correct trigger area), so it all just works. Here's that in action with the extra wide timing windows:



[h2]Ghost Helpers[/h2]

I got to see and/or hear about a variety of play experiences when I did my last internal playtest. I won't lie: the design of Rhythm Quest is almost entirely driven by my own sensibilities, but it's still useful to see other perspectives so I can make small course-corrections and admissions when I feel necessary.

Interestingly (or maybe this shouldn't really be surprising), different people seemed to struggle more or less with different aspects of the game -- for some, the water zones really threw off their ability to read the charts, while other people mentioned the ghost enemies being difficult to read since they felt like they needed to react to each of the new positions.

For people who struggle with the ghost enemies, I've added a new helper option that will display guidelines for the second and third positions of the ghosts:



This does, of course, take away from the entire conceit of the ghost enemies to begin with (with the helpers, it's no different than three red enemies), but I really don't mind adding these sorts of modifiers when the implementation is so simple (and I'm not busy working on other things). You can play Rhythm Quest however you want!

[h2]Performance Settings[/h2]

I already did a bunch of work on performance optimizations in a previous stint of work, but I'm happy to report that I've also made the rendering of the water sections more performant: previously, the render textures used to implement the "wavy" water shader were way bigger than they needed to be...now they don't extend past the height of the screen, and dynamically shift based on the camera to be more efficient.

However, I also wanted to give a nod to some lower-end devices, and ended up adding a graphics quality toggle that will disable the fancy water shader entirely, as well as cut out some of the translucent backdrop layers (that I pick by hand) in an attempt to make rendering more performant. I also added another engine-level setting for controlling frame queueing. Those all live on a new sub-settings page that comes complete with a quick-and-dirty FPS counter and some water zones in the background so you can see the effects of the settings live:



[h2]Other Stuff[/h2]

Some miscellaneous other stuff was added too, like a "screenreader" command-line flag that will force the game to start in screenreader mode even if it doesn't auto-detect that one is enabled.

In an attempt to add a little more discoverability to the shop menu, I also made that accessible from the "Extras" menu in addition to the normal place in the level select screen. I also renamed "Cheats" as "Game Mods" since I'm throwing in all of the timing and visual modifications that I've added there. Most of those are also accessible via the in-game menu, which can be helpful if you need a temporary assist for a tough checkpoint that you're struggling on.



I think that about covers it for what I've been working on lately!

I'm continuing my break from working on world 6 levels for now. I've sent out the current build to some internal beta testers to get some feedback on the difficulty scaling and reception to the newer mechanics, so I want to give myself a chance to let that feedback come in and stew on it for a bit before I continue on with the last 4 levels of the game.

In the meantime, I've been trying to tackle some improvements and fixes that have been laying around in my backlog for a while...

[h2]Gamepad Rebinds[/h2]

This one has been desired (and requested) a long while ago, but I kept on putting it off because I wasn't sure exactly how I wanted to handle it.

I already had control rebindings working just fine for keyboard controls, which have a single key assigned to each action:



The problem with gamepad bindings is that by default the gamepad controls have many different bindings: to jump you can use the d-pad, the left analog stick, the south or east face buttons, or even the left shoulder button.

I was sort of at a loss for how to deal with this, both in terms of the UI (how to show the combined default bindings?) and in terms of implementation (how to override the entire set of bindings at once?).

Like many other tricky problems I've run across in Rhythm Quest, letting it sit in the back of my head for a while allowed me to come up with a different approach:



Gamepad and keyboard bindings now each have their own standalone submenu (not available on platforms where they don't apply). More importantly, there's an individual setting that toggles between a "default binds" set and a "custom binds" set. The default binding set features multiple binds, whereas the custom binding set only has two (that can be overriden by the user). This elegantly (?) solves the issue I mentioned above.

This also lets me illustrate the controls in a hand-drawn diagram, something that's probably easier to parse than "Jump: DPad, Left Stick, Left Shoulder, A, B, ..."

Using the same system, I'm even able to detect whether a (supported) gamepad is plugged in at all, and dynamically update the screen accordingly:



I adopted the same tech for the keyboard bindings screen as well (had a bit of fun trying to draw a keyboard layout):



You'll notice that I decided to also expand the default bindings to just encompass the entire left/right half of the main keyboard keys. Unity does a reasonably good job (?) of detecting keys based on physical location, so this should work even if you use a nonstandard key layout like I do. I'm not sure what will happen for non-ANSI physical layouts, but I'm assuming the custom binding system will suffice for any odd edge cases.

For now I'm providing two custom binding slots for each action (an improvement over before where you could only use one key), in case you want to alternate keys for faster sections.

As usual, there's a ton of silly little details that need to be handled with input rebindings, and as usual, Unity provides just enough functionality to be helpful, but also forces you to work with a ton of abstractions like "InputControlPaths", "InputActions", and "ControlSchemes" that end up making your head spin when you think about them too much. You need to, for example, make sure that a rebinding can be cancelled via either Gamepad OR Keyboard input (the input system by default only allows you to define a single cancellation binding...)...

[h2]Rendering Artifacts[/h2]

This is a really silly one, the kind of thing that you'd never imagine would be an issue, but somehow it is. Rendering the game to a width or height that's an odd number (e.g. 1013x533) causes weird visual artifacts:



This is caused by camera scaling and such -- here, the resolution is 501x301 and the game has decided to render the pixels at 2x, which means the base resolution is 250.50x150.50, which doesn't work out too nicely.

I tried to address this before by automatically resizing the game window and forcing it to be a multiple of two, but that didn't work too well. My new solution is to handle the rendering properly by shifting the camera by a similar fractional amount, so here we simply shift the camera over by a half pixel and fortunately that works to fix things.

[h2]Released Early/Late[/h2]

Suggested by one of my playtesters -- the "Too Early / Too Late" text for holds is now more specific in calling out "Released Early / Released Late". A super easy fix that hopefully helps clarity a tiny bit:



I'm glad I got around to some of these improvements and fixes (which should be coming to the demo soon), but I feel like I've only just scratched the surface of the work that needs to be done. Even for the gamepad rebinding system, I still need to test how it works on Switch / for other types of gamepads, and could even stand to draw different graphics (especially for the Switch joycons). There's also some tweaks that I'm going to be trying to look at after seeing how playtesters fared with the current build...

The year is about halfway over and unfortunately my progress hasn't been super great -- I've only managed to finish off 6 levels in that time, plus some optimization work/etc. Of course, I had some real life stuff happen that drew my attention away, but that's also sort of true in the upcoming months as I help mentor for a video game tournament. That "end of 2023" date is starting to feel really scary when I think about it...

The Rhythm Quest Demo has been updated to version 0.26.5! This patch includes some hefty optimizations, which should help improve performance, particularly on lower-end machines.

Full changelog:

Version 0.26.5
​- Reworked texture encoding for memory and performance benefits
​- Various other performance optimizations
​- Fixed an issue causing minor vertical blurring on WebGL builds
​- Add better transition for Furball attack -> jump
​- Fixed attack animation being cancelled on fly start
​- Fixed menu bug where foreground level backdrops sometimes failed to fade in
​- Minor UI fixes

Somewhat unexpectedly, I took a break from working on levels this month to focus instead of **performance and memory optimizations**. This was brought on by the fact that I made some release builds for the first time in a while and found that my iOS build crashed on startup because it was running out of memory loading the main menu!

The main culprit? These huge backdrop texture atlases...(this one is 64 MB!)...



[h2]The Problem[/h2]

Your first thought upon seeing these atlases is that they're really wasteful. Why is there so much empty space in the upper-right? Well, that one is because the texture atlases need to be even powers of 2 in dimensions (1024, 2046, 4096). I could, of course, have each layer be separate, without packing them into a single atlas, but then I'd lose all the performance benefits of being able to batch the draw calls for all of the background layers together.

The better question is why does each backdrop layer have so much vertical padding? Well, I didn't want to make any assumptions about the player's aspect ratio, resolution, or zoom settings, so the easiest way for me to solve that was to just author all of my backdrop layers with loads of vertical leeway, so that they'll always be fully in view.



Each separate layer is exported at 500x1200 pixels (very tall!), and then tiled horizontally by the game. Some of the levels have upwards of 10 or 15 separate backdrop layers, so that's quite a lot of pixels...

[h2]Texture Encoding[/h2]

The first thing I wanted to do was see if I could just store the textures more efficiently without changing anything about my authoring workflow. You may have noticed that the texture atlases are all grayscale (no color). This is a change I made a long time ago, back when I decided to use a palette shader for the backdrops. Essentially, I only really need to represent indices into my color palette (currently, one of 11 colors), so during my export I just use grayscale colors that the pixel/fragment shader can read and then interpret as color 0, color 1, etc. I also sometimes have partial transparency, so the alpha value is also important.

However, the textures are still encoded as 32-bit RGBA, which means 8 bits are assigned to each of the red, green, blue, and alpha channels! That's pretty wasteful, so I wanted to look into whether Unity supports other lossless texture formats (across multiple platforms). It does, in fact you can actually use the "R 8" texture format, which exclusively encodes a red channel (nothing else!), and only uses 8 bits per pixel (25% of what I was currently using!).

That seemed perfect, as really all I needed was grayscale values anyways. The one problem was that I still needed to store alpha values to handle partial transparency. Could I somehow pack both the color index, and the alpha information, into 8 bits?

Since I only have 11 different colors in my color index, 4 bits is enough to encode that (2^4 = 16). That would leave the other 4 bits to store alpha information, which would mean I could have 16 different possible alpha values. That's more than enough for my purposes, so I went ahead with this strategy of using 4 bits for color encoding and the other 4 bits for alpha values:



To get this all working, I needed to first write a python script to take all of my original backdrop exports and encode them into an 8-bit red channel like you see above. Then I needed to modify my palette shader to do the reverse: take the 8-bit encoding and parse it into a color index and an alpha value.

After a bunch of shader math debugging and fussing around with bit arithmetic, it was all working (everything looked the same as before) and the iOS build was no longer crashing. Hooray!

[h2]Texture Cropping[/h2]

We can still do better, of course. The next step was to see if I could get rid of all of the extra padding on the top and bottom of many of these images. Take this cloud layer for instance:



Ideally we could only store the actual texture data that really matters (the middle section). The top half is all transparent, so we can just discard that, and then for the bottom half we can just "clamp" the texture lookup so that the bottom-most opaque row is essentially repeated indefinitely.

Doing the crop itself is simple enough -- I just modify my python image-processing script to analyze the rows of the image and trim it accordingly. We end up with this nice cropped version of the image:



The trickier part is that we now need to render this in the same way as the original texture. There are a couple of problems with this...

First, the new origin/center point of the sprite is different than before, since we trimmed an unequal amount of rows from the top and bottom, so it's going to be offset from where it was supposed to be drawn. To fix this, I added processing to my script to keep track of how much the new cropped sprite is offset by. I also track some other important metadata, such as whether the top or bottom sections (or both) should be repeated transparency, or a repeated opaque row. Then I output that all to a C# file that I can read in:

{ "level2-5_background_4", new Entry {
    Offset = -62.5f,
    TopTransparency = true,
    BottomTransparency = false,
    OpaqueBelow = 1,
    OpaqueAbove = 55
} },

My backdrop tiling script is responsible for taking the stored offset metadata and shifting the center position of the rendered sprite accordingly.

The second issue is that while Unity supports texture coordinate clamping, there's no way to do that when the sprite in question is one of many sprites packed into a texture atlas! Unity's sprite renderer only handles tiling in a very specific way, which no longer applied to what I wanted to do, so I had to modify my fragment shader to handle the texture clamping part.

In order to do this texture clamping correctly, I also needed my fragment shader to understand what UV texture coordinates it was supposed to be working with inside the texture atlas. Normally the fragment shader is completely oblivious of this -- the Sprite renderer is responsible for handing it a set of UVs to render and then the shader just does the texture lookups blindly.

It also turns out that you don't actually have access to the sprite UV metadata from within your fragment shader =/. So I needed to pass those into the shader, =and= I couldn't use uniform variables since that would break batching. Luckily, Unity happens to expose a SpriteDataAccessExtensions class which allows you to write to the UV texture coordinates of the sprite mesh used by a sprite renderer internally.

In addition to allowing you to modify the main UVs, it also lets you set additional texture coordinates on the mesh (TexCoord1, TexCoord2, TexCoord3, etc.). I used those to pass extra data to the vertex shader -- and then through to the fragment shader -- including the sprite UVs from the texture atlas.

This took a lot more debugging to get right, but at the end of all that, it was working! Here's the new version of the texture atlas from before (in all its red-channel glory), which is 1024x1024 instead of 4096x4096, and 1 MB instead of 64 MB!



[h2]Alleviating Overdraw[/h2]

Rhythm Quest isn't really a performance-intensive game, so it runs fine on most systems. That said, there are a couple of areas where it can get into performance issues on lower-end devices (surprisingly, the Nintendo Switch is the main culprit of this so far).

One major performance bottleneck involves overdraw, which is a term used to describe when pixels need to be rendered multiple times -- typically an issue when there are many different transparent / not-fully-opaque objects rendered in the same scene (*cough* backdrop layers *cough*).

Unlike in a generic 3d scene (where we would try to render things from front-to-back, to minimize overdraw), for our backdrop layers we need to render things from back-to-front in order to handle transparency correctly:



Unfortunately, this results parts of the screen being rendered to many times over and over again, particularly the lower areas (all of those overlapping cloud layers...). The good news is that the cropping we did above already does some work to alleviate this a bit. Before, the large transparent portions of backdrops would still need to go through texture lookups and be rendered via the fragment shader, even though they were completely transparent (i.e. didn't affect the output). But now, we've cropped those areas out of the sprite rendering entirely, so they aren't a concern.

We can still do a little more optimization, though, for opaque backdrop sections! Take this layering of opaque cloud layers from level 2-5 as an example:



There's a lot of overdraw happening on the bottom sections of the screen. What if we were smart about this and kept track of which portions of the screen are being completely covered by each layer, front-to-back? That would let us render smaller screen sections for all of the back layers:



We can handle this by having our image processing script store some additional metadata (the "OpaqueBelow" and "OpaqueAbove" fields) so we know at which point a background layer obscures everything above or below it. We then need to modify the backdrop script to adjust the drawing rect and UVs accordingly (easier said than done...)...

The end result of all of this is...that everything looks exactly the same as before...



But! It's significantly more efficient both in terms of memory usage and rendering time. I'll have to patch the existing demo builds with this optimization at some point, but the Switch build is already showing some improvements, which is nice.

We're not completely done with performance though, as right now the rendering of the water sections are also quite inefficient! I may try to tackle that next...

I'm continuing to just roll ahead with levels! It's funny, I feel like there was a long period of time when working on new levels and thinking about the mechanics felt intimidating, so I would just procrastinate on it and work on other miscellaneous things. But now I think it's the opposite (probably partly because all of my mechanics are known now), where I've gotten into the habit of just working on only levels. It's good though, the levels are something that need to be done 100%.

Anyways, I went straight ahead and finished up the first level in world 6, level 6-1!

[previewyoutube][/previewyoutube]

[h2]Speed Zones[/h2]

World 6 introduces one new mechanic, the red "speed zones" that increase scroll speed and change up the rhythmic meter into triplet patterns (quarter note triplets) temporarily:



As with some of my other mechanics, this might get mixed initial reactions from players (or at least, that's the expectation I'm setting up for myself...). For people who don't "get" triplet meter, it might seem sort of like an arbitrary changeup/speedup that's hard to react to. I experimented with having a sort of 2-beat "lead-in" to prep you for the new meter, but I was pretty unhappy with how that sounded (messy...) so I took it out. (Maybe that'll be an optional toggle someday?)

For now I'm just trying to give the player some easy speed zones at first so that they can listen to and get used to the rhythm, before I throw actual quarter-note triplets at them:



You might not have noticed it until I pointed it out (now it'll stick out like a sore thumb...), but none of the speed zones have any height ramps -- they're all completely flat. I couldn't really get the "conveyor belt" graphic to look reasonably good at an angle, so I just decided to add that as a restriction (the level generator will probably be really confused if you try to add ramps in the middle of it). I'm totally ok with that though, it makes them simple to read...and I actually have the same restriction for spike enemies (they can travel across ramps, but the actual jump needs to be on flat ground), so it's not really a new thing. I guess technically I can support height changes in the form of air jump combos and flight paths, but those haven't come up yet.

[h2]Visual Identity[/h2]

This one was easy since I had already been thinking for a long time to do an outer space theme for world 6 (maybe sort of a trope to have the final area be space-themed?). One of the worries here is that all of the level backdrops are just going to look similar since they'll all just be dark skies with stars, but hopefully I can make them a little bit distinct by experimenting with different foreground elements and such.



For this level I went with sort of a "spiral galaxy"-type drawing with a bright orb in the middle. In hindsight, I probably could have drawn it bigger...but I guess this way it's more of a single element rather than filling most of the screen, which works too. It looks like there's all sorts of colors in there, but it's really just the 8-color palette, but with a bunch of translucent layers. It was actually quite fun to draw, as it felt like more of a painterly (impressionistic?) approach throwing blobs and dots of colors everywhere rather than the geometric shapes from world 5. You can also see that I'm making heavy use of the spraypaint tool for the first time here, particularly in the soft "nebula"-like patterns in the background.

As usual, I tried to add in some amount of variation in the color palette depending on the different sections of music. Here I switch to a completely black background color for the first "main" section of the song to up the contrast level a little bit:



I'm hopeful about this art style for world 6! Hopefully I'll be able to draw some nice backdrops by experimenting with this general direction. I was a bit worried at first since I feel like "space" art tends to not do well with such limited color palettes, but it's turning out fine with clever use of dithering-like effects and translucency.

[h2]Musical Identity[/h2]

Unlike with world 5, I didn't do a whole ton of musical exploration before starting off on this level...I sort of just "winged it" and went with some rough ideas, seeing what came out of them. I knew I wanted to try playing around with whole-tone scale melodies, but I was also interested in exploring more varied bass sounds (maybe even dubstep-esque), as well as featuring prominent use of arpeggios and low-pass/high-pass filter automation.

Here's a snippet showcasing the "wub" bass featured in this track, as well as a triangle wave synth that plays a whole tone scale pattern. I dunno, somehow wobbly basses and triplet rhythms almost seems like a bit of a musical trope...

https://rhythmquestgame.com/devlog/53-wholetoneandbass.mp3

Here's another snippet, showing off some low-pass filter automation on gated chords, as well as an arpeggio that has some long reverb on it (spacey!).

https://rhythmquestgame.com/devlog/53-arpfilter.mp3

And here's a longer snippet of the main buildup in the song. I use a different (but still-prominent) bass here, and slowly open up the filter as it builds. As with world 5, I'm making heavy use of triangle-wave tom fills to accentuate the rhythmic changeups.

https://rhythmquestgame.com/devlog/53-buildup.mp3

[h2]Level Select[/h2]

A new world also means a new level select theme! Here's a short video where you can hear that in action:

[previewyoutube][/previewyoutube]

I had a few false starts on this one before I landed on the idea, but it sounds great! I love how the major IV -> minor iv progression works here. You can hopefully hear the low-pass filter automation on the chorded synth, as well as the reverbed short arpeggio pattern -- same ideas as in the level.

There's still a bunch more to explore with speed zones and how they combine with the other mechanics, which should be interesting to figure out over the course of these next 4 levels! I might have to tread a tricky balance since fast rhythms (e.g. double-hit enemies) are =really= fast in speed zones, so those will only be feasible if I take the overall tempo down a notch...