Console rework: ANSI compliance; visible cursor

[trustytrojan]

Changes

• Refactored console code into three main components:
  • Tile rendering: console-print.c
  • Escape sequence handling: console-esc.c
  • Cursor movement: console-cursor.c
• console-priv.h was added as an interface for the three source files to call each other with.
• This also fixes the bug presented in Fix read out-of-bounds when accumulating escape sequence #72

Feature additions

• Be compliant with all ANSI color/graphics modes
• Be able to parse 256 color codes, only applying values 0-15
• Be able to parse RGB color codes, currently ignoring their parameters.
• Have a visible cursor, even when moving over tiles with characters!
  • Only with true passed to ansiBgColors in consoleInit

Implementation details

Because the default font cannot have tiles inverted in color (or I just don't know how to do that), I used the next available video background layer (new parameter added in consoleInit documentation) just for rendering the ASCII full block character (decimal 219) using the same palette.

By adding a simple boolean parameter to consoleInit, you can still choose whether a console uses 2 BGs or not.

Demo video

recording_2025-08-08_16.55.50.mp4

Demo code

#include <nds.h>

#include <iostream>
#include <print>

int main()
{
	defaultExceptionHandler();

	// Video initialization - We want to use both screens
	videoSetMode(MODE_0_2D);
	videoSetModeSub(MODE_0_2D);
	vramSetBankA(VRAM_A_MAIN_BG);
	vramSetBankC(VRAM_C_SUB_BG);

	// Show console on top screen
	static PrintConsole console;
	consoleInit(&console, 0, BgType_Text4bpp, BgSize_T_256x256, 20, 0, true, true, true);

	keyboardDemoInit()->scrollSpeed = 0;
	keyboardShow();

	std::cout << "hello world\n\n1234567890\n\nABCDEFGHIJKL\n\n";

	static const char *const colorstr[] = {"black", "red", "green", "yellow", "blue", "magenta", "cyan", "white"};

	for (int j = 0; j <= 7; ++j)
		std::println(
			"\e[39m\e[3{}m{:<8}\e[39m\e[4{}m{:<8}\e[39m\e[9{}m{:<8}\e[39m\e[10{}m{:<7}\e[39m",
			j,
			colorstr[j],
			j,
			colorstr[j],
			j,
			colorstr[j],
			j,
			colorstr[j]);

	while (pmMainLoop())
	{
		swiWaitForVBlank();
		switch (const s8 c = keyboardUpdate())
		{
		case NOKEY:
			break;
		case DVK_UP:
			std::cout << "\e[A";
			break;
		case DVK_DOWN:
			std::cout << "\e[B";
			break;
		case DVK_RIGHT:
			std::cout << "\e[C";
			break;
		case DVK_LEFT:
			std::cout << "\e[D";
			break;
		default:
			std::cout << c;
			break;
		}
	}
}

[WinterMute]

This is one of the reasons we ask people to contact us before embarking on large changes to the libraries we maintain. We've been doing work on reworking the ansi support code in all the libraries which we've been consolidating, bugfixing and cross porting. See https://github.com/devkitPro/libogc/blob/master/libogc/console.c, https://github.com/switchbrew/libnx/blob/master/nx/source/runtime/devices/console.c and https://github.com/devkitPro/libctru/blob/master/libctru/source/console.c

I'm sorry you've put so much work into this but libnds was next for this treatment and your changes unfortunately conflict with that.

[trustytrojan]

That's alright, I got what I wanted out of this anyway. I just hope the result is the same after you guys get to libnds. Though I have to ask, how will you get background colors to work without using a 2nd layer? Unless that was your plan. That's probably the change with the largest effect on users so I'm wondering. I didn't want to deal with custom fonts so a 2nd layer was my solution.

Edit: here's something fun I got out of this work: star wars turned into ascii art streamed over telnet streamed over ssh!

recording_2025-08-11_16.29.13.mp4

Edit 2: I just remembered that there's a PrintChar callback field on the PrintConsole struct. My work is definitely not for nothing if I can just hook my logic right into that!