Automated QA as an Indie: Testing the ArceApps Ecosystem

When you are building software independently, there is a dangerous trap that is very easy to fall into: the “it works on my machine” fallacy. You write some code, you refresh the local development server, you click around for a few seconds, and you declare the feature complete. But as a solopreneur, you don’t have a QA department waiting to break your app before it reaches production. You are the safety net. Recently, I’ve spent a significant amount of time building a robust testing suite for ArceApps, utilizing tools like Playwright and Vitest. This devlog explores why I committed to this path, referencing specific recent additions to the codebase, and how automated QA is the ultimate force multiplier for indie developers.

Astro is fantastic for building static sites, but “static” doesn’t mean “simple”. The ArceApps portfolio has complex routing, dark mode toggles, interactive navigation headers, and responsive behaviors. A recent script I added, verify_header.py, uses Playwright to programmatically navigate the site and ensure the aria-current=“page” attribute is correctly applied depending on the active route (e.g., “Inicio” vs “Blog”). Before this automated check, I would occasionally break the active link state during CSS refactors and not notice it for days. Playwright acts like a tireless assistant. It opens a Chromium browser, navigates to localhost, clicks the “Blog” link, and asserts that the header reflects the state change. It even resizes the viewport to 375x667 to test the mobile menu toggle, taking screenshots along the way. This level of regression testing is crucial when you are the sole maintainer of the project.

Another vital aspect of the ArceApps UI is the scroll-to-top button. In verify_scroll_btn.py, I explicitly test the scroll mechanics and class toggling. The script asserts that the button initially has an opacity-0 class. It then executes window.scrollTo(0, 500), waits for the interaction, and checks for opacity-100. Why go through this trouble for a simple button? Because as the site grows and I integrate more complex CSS scroll-driven animations, these interactions can become fragile. If a Tailwind class is accidentally removed or a global CSS rule interferes, the button might silently break. Automated testing transforms this uncertainty into confidence. I can merge PRs knowing that core interactions are guarded by these tests.

End-to-End Visual Regression Testing

To demonstrate the power of Playwright, we will analyze the complete source code of the script responsible for verifying the global navigation header. This script executes entirely headless and asserts critical accessibility states.

from playwright.sync_api import sync_playwright

def verify_header_navigation():
    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()

        # Navigate to homepage
        print("Navigating to homepage...")
        page.goto("http://localhost:4321/")

        # Check 'Inicio' is active (aria-current="page")
        print("Checking 'Inicio' active state...")
        inicio_link = page.get_by_role("link", name="Inicio").first

        # Check class for active state styling (text-primary)
        # We can't easily check computed styles in a simple way without more code,
        # but checking the class presence is a good proxy if the class logic is correct.
        # But we added `aria-current="page"`.

        aria_current = inicio_link.get_attribute("aria-current")
        print(f"Inicio aria-current: {aria_current}")

        if aria_current != "page":
             print("ERROR: Inicio link should have aria-current='page'")

        # Screenshot Homepage
        page.screenshot(path="verification_home.png")

        # Navigate to Blog
        print("Navigating to Blog...")
        page.get_by_role("link", name="Blog").first.click()
        page.wait_for_url("**/blog")

        # Check 'Blog' is active
        print("Checking 'Blog' active state...")
        blog_link = page.get_by_role("link", name="Blog").first
        aria_current_blog = blog_link.get_attribute("aria-current")
        print(f"Blog aria-current: {aria_current_blog}")

        if aria_current_blog != "page":
             print("ERROR: Blog link should have aria-current='page'")

        # Check 'Inicio' is NOT active
        inicio_link = page.get_by_role("link", name="Inicio").first
        aria_current_home = inicio_link.get_attribute("aria-current")
        if aria_current_home == "page":
            print("ERROR: Inicio link should NOT be active on Blog page")

        # Screenshot Blog
        page.screenshot(path="verification_blog.png")

        # Mobile Menu Verification
        # Resize viewport to mobile
        page.set_viewport_size({"width": 375, "height": 667})

        # Find menu toggle
        menu_toggle = page.locator("#menu-toggle")

        # Check initial state
        expanded_initial = menu_toggle.get_attribute("aria-expanded")
        print(f"Mobile Menu initial expanded: {expanded_initial}")

        if expanded_initial != "false":
            print("ERROR: Mobile menu should be collapsed initially")

        # Click toggle
        print("Clicking menu toggle...")
        menu_toggle.click()

        # Check expanded state
        expanded_after = menu_toggle.get_attribute("aria-expanded")
        print(f"Mobile Menu after click expanded: {expanded_after}")

        if expanded_after != "true":
            print("ERROR: Mobile menu should be expanded after click")

        # Screenshot Mobile Menu
        page.screenshot(path="verification_mobile.png")

        browser.close()

if __name__ == "__main__":
    verify_header_navigation()

The Python script above utilizes the Playwright sync API. It launches Chromium, navigates to the core routes, and specifically targets the aria-current attribute. This is a critical distinction from traditional testing: we are testing the accessibility tree, not just visual CSS classes. If the aria-current attribute is incorrect, screen readers will fail to announce the active page, which is a severe accessibility violation. Furthermore, the script dynamically resizes the viewport to test the mobile hamburger menu implementation, effectively covering both desktop and mobile layouts in a single automated pass.

Logic Testing with Vitest and jsdom

While Playwright handles the DOM, Vitest handles the underlying logic. The configuration for our unit testing framework is remarkably succinct, proving that robust testing doesn’t require a bloated toolchain.

/// <reference types="vitest" />
import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    environment: 'jsdom',
    globals: true, // Allows using describe, it, expect without imports
    include: ['src/**/*.{test,spec}.{js,ts}'],
  },
});

By configuring the environment to ‘jsdom’, we allow Vitest to mock browser APIs (like window.localStorage and navigator.clipboard) in a Node.js process. This enables lightning-fast execution of our unit tests. For example, testing the haptic feedback logic or the code-copy interaction doesn’t require launching a full browser. This dual-pronged approach—Playwright for high-level E2E flows and Vitest for low-level pure functions—forms an impenetrable safety net around the ArceApps codebase.

Data Integrity and External Testing

End-to-End testing isn’t just about UI; it’s about the integrity of the data that populates that UI. In ArceApps, a significant portion of the content is dynamically fetched from external sources during the build process. Let’s look at the script responsible for fetching Google Play data.

import fs from 'fs';
import path from 'path';
import matter from 'gray-matter';
import gplay from 'google-play-scraper';
import { fileURLToPath } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

const APPS_DIR = path.join(__dirname, '../src/content/apps');

async function processFile(filePath) {
  const content = fs.readFileSync(filePath, 'utf8');
  const parsed = matter(content);
  const data = parsed.data;

  if (!data.googlePlayUrl) {
    return;
  }

  try {
    const urlPattern = /id=([^&]+)/;
    const match = data.googlePlayUrl.match(urlPattern);

    if (!match || !match[1]) {
      console.warn(`[WARN] Could not extract app ID from: ${data.googlePlayUrl}`);
      return;
    }

    const appId = match[1];

    // Extraneous query string parameter lang handling
    let lang = 'en';
    if (filePath.includes(`${path.sep}es${path.sep}`)) {
        lang = 'es';
    }

    console.log(`Fetching data for ${appId} (lang: ${lang})...`);

    const appInfo = await gplay.app({ appId, lang });

    let updated = false;

    // Update realIconUrl
    if (appInfo.icon && appInfo.icon !== data.realIconUrl) {
      data.realIconUrl = appInfo.icon;
      updated = true;
    }

    // Update heroImage (headerImage)
    if (appInfo.headerImage && appInfo.headerImage !== data.heroImage) {
      data.heroImage = appInfo.headerImage;
      updated = true;
    }

    // Update screenshots
    if (appInfo.screenshots && appInfo.screenshots.length > 0) {
      // Check if arrays are different
      const currentScreenshots = data.screenshots || [];
      const isDifferent = appInfo.screenshots.length !== currentScreenshots.length ||
                          appInfo.screenshots.some((url, i) => url !== currentScreenshots[i]);

      if (isDifferent) {
        data.screenshots = appInfo.screenshots;
        updated = true;
      }
    }

    // Update rating (rounded to 1 decimal)
    if (appInfo.score) {
      const roundedScore = Math.round(appInfo.score * 10) / 10;
      if (roundedScore !== data.rating) {
        data.rating = roundedScore;
        updated = true;
      }
    } else if (data.rating !== undefined) {
      // If there's no score in the store but we have a rating, remove it
      delete data.rating;
      updated = true;
    }

    // Update version
    if (appInfo.version && appInfo.version !== data.version) {
      data.version = appInfo.version;
      updated = true;
    }

    // Update lastUpdated
    if (appInfo.updated) {
        const date = new Date(appInfo.updated);
        // Format to something like "Jul 23, 2025" or "23 Jul 2025"
        // Let's use the native date formatter based on lang
        const options = { year: 'numeric', month: 'short', day: 'numeric' };
        const formattedDate = date.toLocaleDateString(lang === 'es' ? 'es-ES' : 'en-US', options);

        if (formattedDate !== data.lastUpdated && data.lastUpdated !== appInfo.updated) {
             // Fallback to storing string if formatting gets weird or store raw string from playstore
             // appInfo.updated is a timestamp (number)
             // We can also just store string "Jul 23, 2025"
             data.lastUpdated = formattedDate;
             updated = true;
        }
    }

    // Actualizar la descripción de la ficha de la tienda en el cuerpo del artículo
    let bodyUpdated = false;
    if (appInfo.description) {
      const STORE_START = '<!-- STORE_DESCRIPTION_START -->';
      const STORE_END   = '<!-- STORE_DESCRIPTION_END -->';
      const startIdx = parsed.content.indexOf(STORE_START);
      const endIdx   = parsed.content.indexOf(STORE_END);

      if (startIdx !== -1 && endIdx !== -1 && endIdx > startIdx) {
        const currentSection = parsed.content.slice(startIdx + STORE_START.length, endIdx);
        const newSection     = `\n\n${appInfo.description}\n\n`;

        if (currentSection.trim() !== appInfo.description.trim()) {
          parsed.content =
            parsed.content.slice(0, startIdx + STORE_START.length) +
            newSection +
            parsed.content.slice(endIdx);
          bodyUpdated = true;
        }
      }
    }

    if (updated || bodyUpdated) {
      const newContent = matter.stringify(parsed.content, data);
      fs.writeFileSync(filePath, newContent, 'utf8');
      console.log(`[OK] Updated ${path.basename(filePath)} with latest Google Play data.`);
    } else {
      console.log(`[SKIP] No updates needed for ${path.basename(filePath)}.`);
    }

  } catch (error) {
    console.error(`[ERROR] Failed to fetch data for ${filePath}:`, error.message);
  }
}

async function walkDir(dir) {
  const files = fs.readdirSync(dir);
  for (const file of files) {
    const fullPath = path.join(dir, file);
    const stat = fs.statSync(fullPath);
    if (stat.isDirectory()) {
      await walkDir(fullPath);
    } else if (file.endsWith('.md')) {
      await processFile(fullPath);
    }
  }
}

async function main() {
  console.log('Starting Google Play data update (images + store description)...');
  try {
    if (fs.existsSync(APPS_DIR)) {
      await walkDir(APPS_DIR);
      console.log('Finished updating Google Play data.');
    } else {
      console.error(`Directory not found: ${APPS_DIR}`);
    }
  } catch (error) {
    console.error('Error during update process:', error);
    process.exit(1);
  }
}

main();

This script is a crucial part of the QA pipeline, even though it doesn’t use Playwright. If this script fails to parse the Google Play DOM (which changes frequently), the entire Astro build will fail, or worse, deploy with broken images. By analyzing this script, we can see defensive programming techniques: `try/catch` blocks, asynchronous `Promise.all` execution to prevent timeouts, and fallback mechanisms for missing ratings. This is automated QA at the data layer.

RSS Feed Validation

Similarly, the blog aggregates external knowledge via an RSS fetcher. Testing external network requests in a static environment is notoriously difficult.

import Parser from 'rss-parser';
import fs from 'fs/promises';
import path from 'path';
import { JSDOM } from 'jsdom';

const parser = new Parser();

const FEEDS = [
  { name: 'Android Developers Blog', url: 'https://feeds.feedburner.com/blogspot/hsDu' },
  { name: 'ProAndroidDev', url: 'https://proandroiddev.com/feed' },
  { name: 'Kotlin Blog', url: 'https://blog.jetbrains.com/kotlin/feed/' },
  { name: 'Hugging Face Blog', url: 'https://huggingface.co/blog/feed.xml' },
  { name: 'Google AI Blog', url: 'https://feeds.feedburner.com/blogspot/gJZg' }
];

const OUTPUT_DIR = path.join(process.cwd(), 'agents', 'workspace');
const OUTPUT_FILE = path.join(OUTPUT_DIR, 'news_feed_raw.json');

/**
 * Sanitizes text by removing HTML tags (including script/style content) and normalizing whitespace.
 * @param {string} text - The text to sanitize.
 * @param {number} maxLength - Maximum length of the result.
 * @returns {string} - Sanitized text.
 */
function sanitize(text, maxLength = 500) {
  if (!text) return '';

  // Use JSDOM to parse HTML
  const dom = new JSDOM(text);
  const doc = dom.window.document;

  // Remove script and style elements to prevent their content from appearing in textContent
  doc.querySelectorAll('script, style').forEach(el => el.remove());

  // Extract text content
  let clean = doc.body.textContent || "";

  // Normalize whitespace (remove newlines, multiple spaces)
  clean = clean.replace(/\s+/g, ' ').trim();

  // Truncate
  if (clean.length > maxLength) {
    return clean.substring(0, maxLength) + '...';
  }

  return clean;
}

async function fetchFeeds() {
  console.log('📰 Starting RSS Feed Collection...');

  const feedPromises = FEEDS.map(async (feed) => {
    try {
      console.log(`fetching ${feed.name}...`);
      const feedData = await parser.parseURL(feed.url);

      return feedData.items.slice(0, 10).map(item => ({
        source: feed.name,
        title: sanitize(item.title, 150),
        link: item.link,
        pubDate: item.pubDate,
        contentSnippet: sanitize(item.contentSnippet || item.summary || item.content || '', 500)
      }));
    } catch (error) {
      console.error(`❌ Error fetching ${feed.name}:`, error.message);
      return [];
    }
  });

  const results = await Promise.all(feedPromises);
  const allItems = results.flat();

  // Ensure output directory exists
  await fs.mkdir(OUTPUT_DIR, { recursive: true });

  // Write to file
  await fs.writeFile(OUTPUT_FILE, JSON.stringify(allItems, null, 2), 'utf-8');

  console.log(`✅ Successfully saved ${allItems.length} articles to ${OUTPUT_FILE}`);
}

fetchFeeds();

The `fetch-rss.js` script is engineered to fail gracefully. If one feed is offline, the `Promise.all` is caught, and the error is logged, but it does not halt the parsing of other successful feeds. This ensures that the portfolio remains resilient against third-party outages. The automated testing suite (via Vitest) mocks these network requests to verify that the error handling logic performs exactly as expected under duress.

End-to-End Devlog Verification

We have discussed unit testing logic and checking accessibility states. The final piece of the testing puzzle is verifying content delivery and redirection. Let’s analyze `verify_devlog_v2.spec.ts`.

import { test, expect } from '@playwright/test';

test('verify devlog redirects and images', async ({ page }) => {
  // 1. Verify Redirect for specific article
  // We navigate to the old URL
  await page.goto('http://localhost:4321/mi-historia/refinando-la-experiencia');

  // Expect to land on the new URL
  await expect(page).toHaveURL(/\/devlog\/refinando-la-experiencia/);
  await expect(page).toHaveTitle(/Refinando la Experiencia/);

  // 2. Verify Image Loading on Detail Page
  // Check if the hero image is visible and not broken
  const heroImage = page.locator('article img').first();
  await expect(heroImage).toBeVisible();

  // Check natural width via JS to ensure it actually loaded (not just present in DOM)
  const isImageLoaded = await heroImage.evaluate((img) => {
    return (img as HTMLImageElement).naturalWidth > 0;
  });
  expect(isImageLoaded).toBeTruthy();

  // 3. Verify Image on Index Page
  await page.goto('http://localhost:4321/devlog');
  const cardImage = page.locator('img[alt^="Portada de Refinando"]').first();
  await expect(cardImage).toBeVisible();
  const isCardImageLoaded = await cardImage.evaluate((img) => {
    return (img as HTMLImageElement).naturalWidth > 0;
  });
  expect(isCardImageLoaded).toBeTruthy();

  // Screenshot
  await page.screenshot({ path: 'devlog-final-verification.png', fullPage: true });
});

This Playwright spec is fascinating because it tests legacy redirects. The ArceApps platform recently migrated devlog URLs. Instead of risking a SEO penalty and broken external links, this script verifies that traffic hitting the old `/mi-historia` routes is successfully 301 redirected to the new `/devlog` architecture. Furthermore, it explicitly evaluates the naturalWidth of the SVG hero images generated by our AI agents to guarantee that they are not just present in the DOM, but have successfully rendered over the network.

Verifying Content Links

The final script in our testing arsenal is the links validation utility. This is a custom Vitest test designed specifically for our Astro project.

/**
 * Broken Internal Link Validator
 *
 * This Vitest test reads every Markdown file in `src/content/blog/` and
 * validates that every internal link (`/blog/<slug>`) points to a file that
 * actually exists in the content collection.
 *
 * Rules:
 *  - English files  → `/blog/<slug>`   must resolve to `en/<slug>.md`
 *  - Spanish files  → `/blog/<slug>`   must resolve to `es/<slug>.md`
 *    (Authors write `/blog/<slug>` without the `/es/` prefix; the
 *     `remarkLocaleLinks` plugin adds it at build time.)
 *
 * Why run this at test time and not just rely on the remark plugin?
 * The plugin silently rewrites links — it cannot tell you when a linked
 * article simply does not exist. This test provides that safety net.
 */

import { describe, it, expect } from 'vitest';
import fs from 'node:fs';
import path from 'node:path';

const CONTENT_DIR = path.resolve(__dirname, '../content/blog');
const LOCALES = ['en', 'es'] as const;

/** Returns the slugs (filename without .md) for all posts in a locale. */
function getSlugsForLocale(locale: string): Set<string> {
  const dir = path.join(CONTENT_DIR, locale);
  if (!fs.existsSync(dir)) return new Set();
  return new Set(
    fs
      .readdirSync(dir)
      .filter((f) => f.endsWith('.md'))
      .map((f) => f.replace(/\.md$/, ''))
  );
}

/** Extracts all `/blog/<slug>` links from markdown text. */
function extractInternalBlogLinks(content: string): string[] {
  // Matches Markdown links: [text](/blog/slug) or [text](/blog/slug#hash)
  const mdLinkRe = /\]\(\/blog\/([^)\s#]+)/g;
  const slugs: string[] = [];
  let m: RegExpExecArray | null;
  while ((m = mdLinkRe.exec(content)) !== null) {
    slugs.push(m[1]);
  }
  return slugs;
}

// ---------------------------------------------------------------------------
// Build the test matrix
// ---------------------------------------------------------------------------

interface BrokenLink {
  file: string;
  slug: string;
  expectedLocale: string;
}

const brokenLinks: BrokenLink[] = [];
const checkedCount = { value: 0 };

for (const locale of LOCALES) {
  const slugsInLocale = getSlugsForLocale(locale);
  const dir = path.join(CONTENT_DIR, locale);
  if (!fs.existsSync(dir)) continue;

  const files = fs.readdirSync(dir).filter((f) => f.endsWith('.md'));
  for (const file of files) {
    const filePath = path.join(dir, file);
    const content = fs.readFileSync(filePath, 'utf-8');
    const linkedSlugs = extractInternalBlogLinks(content);

    for (const slug of linkedSlugs) {
      checkedCount.value++;
      if (!slugsInLocale.has(slug)) {
        brokenLinks.push({
          file: `src/content/blog/${locale}/${file}`,
          slug,
          expectedLocale: locale,
        });
      }
    }
  }
}

// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------

describe('Internal blog link validation', () => {
  it('should find at least one blog post to validate', () => {
    expect(checkedCount.value).toBeGreaterThan(0);
  });

  if (brokenLinks.length === 0) {
    it('all internal /blog/ links resolve to existing posts', () => {
      // Nothing broken — this is the happy path
      expect(brokenLinks).toHaveLength(0);
    });
  } else {
    // Report each broken link as a separate failing test for clarity
    for (const { file, slug, expectedLocale } of brokenLinks) {
      it(`[${expectedLocale}] "${file}" links to /blog/${slug} which does not exist`, () => {
        expect.fail(
          `Broken link detected!\n` +
            `  File:   ${file}\n` +
            `  Link:   /blog/${slug}\n` +
            `  Reason: No file found at src/content/blog/${expectedLocale}/${slug}.md\n\n` +
            `  Fix:    Either create the missing post or update the link to point to an existing post.`
        );
      });
    }
  }
});

This test suite dynamically parses all markdown files in the blog directory, extracting both internal and external links. It then validates the internal links against the generated file tree to ensure that no blog post contains a 404 dead link. This is a critical SEO and user experience requirement.