At REA I’m part of the Ignite Integrations team, we make it easy for third-party developers to use REA Group’s APIs. Our public-facing developer portal is the source of truth for these developers. It covers how to get credentials, make requests to our APIs, and follow best practices. When the portal is hard to use, support tickets increase and onboarding is slower, so the quality of this site directly shapes how third-party developers experience our APIs.
An 8 year old UI
I don’t typically like to judge a book by its cover, but with websites, sometimes I feel I don’t have a choice. Our developer portal hadn’t changed materially since I joined the team almost two years ago. In fact the main design hadn’t changed in over 8 years. In the footer we had links to social networks that didn’t exist (hello Google+) and brands we’d wound down (farewell Spacely). We were using a version of Hugo (static site generator) that was released in 2017.
The outdated design wasn’t just cosmetic, it was a real distraction from finding information, and it detracted from the credibility of the docs themselves. So when scope was cut on another initiative and some time freed up, I convinced the team it was worth investing in the portal. No more complaining about it!
A fresh coat of paint (and a more maintainable site)
I’d taken some inspiration from Cloudflare about their developer portal upgrade journey and had come across Starlight in the process. Along with feeling more modern, Starlight:
- Supports interactive components written in your framework of choice. We’re most familiar with React, and Astro sets the foundation to one day build a logged-in portal experience with MDX, without wrestling Hugo’s partial syntax for the more complex UI components.
- Comes with batteries included. For free as part of the upgrade we got: sitewide text search, dark mode and composable customisation. Hugo’s model is focused on inheritance where your customisations can collide with upstream theme changes. Starlight’s model is focused on composition where you can “slot” in overriding customisations and still receive updates that don’t conflict.
- Is TypeScript native. Our team and REA more broadly are familiar with and have written plenty of TypeScript, so staying in this ecosystem makes it simple for our team and other teams to contribute to the repo.
I’m not going to dedicate much space to the actual content migration. I asked Claude to migrate the site from Hugo to Astro. I pointed Claude to the documentation of each framework and ensured that metadata embedded in Markdown files was correctly translated to Astro. After a little back and forth with some file path differences and theme customisation, we had an equivalent site running with Astro. Check it out:
As part of this upgrade we enabled renovate. We now get automated dependency updates and in the past month alone, 22 automated dependency update PRs were merged. This ensures we’re kept up to date with any security issues and get access to the latest features of our dependencies. We do have a minimum age set so that the risk of compromised packages being installed is lower. And to reduce the risk of changes breaking the site, we have a suite of automated tests.
Improving the OpenAPI spec rendering experience
One of the biggest pain points with the old developer portal was the API playground. It was broken in many ways. Some pages didn’t work because OpenAPI specs didn’t meet the standard schema. Others didn’t match the implementation of the API. We would get support requests about the playground not working. And honestly, using Swagger to render OpenAPI documents looks dated.
I split this part of the upgrade into a two-step process.
- Improve OpenAPI spec quality and enforce standards.
- Integrate new OpenAPI renderer
We had a mix of Swagger 2.x and OpenAPI 3.x documents at the start, so some manual work was required to convert them all to OpenAPI 3.x. Once that was complete, I set up an automatic step in our build pipeline to lint our OpenAPI documents using the vacuum tool. Along with some generic rules to ensure the specs meet the OpenAPI schema, we also implemented a custom rule that enforces credentials must match a certain pattern in all OpenAPI documents. When downloading or using the OpenAPI documents in the playground, the consistency means that you can use the same set of credentials and it will automatically populate across API definitions.
The second part involved choosing Scalar as a replacement for Swagger. The biggest challenge here was getting the “Test Request” modal opened above the Starlight theme, and automatically toggling dark mode on the rendered OpenAPI document. Fortunately these could be resolved with a little bit of custom CSS. As well as an aesthetics upgrade, Scalar also automatically provides code examples of calling API endpoints in many languages.
Consistency is king
Over time the developer portal accumulated additions and changes, and with no automated way to detect altered URL slugs, broken links were inevitable. Using linkcheck we now have a script that uses Docker Compose to spin up the website in one container and linkcheck in another one and automatically iterates through all links available on the website to ensure they resolve correctly. This script was invaluable during the migration to ensure that all links remained intact (or were fixed if broken before the migration).
Another consistency vector that I was excited to find could be enforced automatically (from the Buildkite docs repo) is prose itself. Vale.sh describes itself as a “command-line tool that brings your editorial style guide to life”. Unfortunately, we don’t have a writing style guide, so the only rule we have is a “spacing” rule that catches double spaces. I was shocked to see it flag 99 instances on its first run. The fix PR was very satisfying.
Future improvements
There’s still work to do though. Like any document, some people would say it’s “living and breathing”. I’d roll my eyes, but would agree with the sentiment. Our APIs and the goals of our users are constantly changing and therefore so should the docs.
Some improvements I’d like to do:
- Implement an automated strategy to identify drift from the OpenAPI description to the actual API implementation. It’s all well and good to have an OpenAPI spec that meets the OpenAPI schema, but if it doesn’t accurately represent the underlying API, you could argue it’s worse than having no spec at all.
- Improve the consistency of language across APIs and domains. Because we have a federated model, teams sometimes diverge on their definitions and terminology. Over time this creates mental overhead and confusion.
- Extend the text searching to the OpenAPI documents. At the moment, because the OpenAPI specs are rendered client-side using Scalar, they can’t be indexed by Pagefind (what Astro uses).
- Optimise the site for LLM usage, including adding clear Markdown file links so that references can be easily ingested or copied into an LLM.
- Add prose linting rules. We should be able to enforce, or at least flag, spelling mistakes, incorrectly stylised words, and other potential stylistic writing issues.
- Simplify information architecture and align pages with the Diátaxis framework. The “Overview” and “Usage” pages feel slightly redundant, and I don’t think “Explore” accurately conveys that the page is a rendered OpenAPI spec. The pages focus on the “Explanation” and “Reference” quadrants from Diátaxis, and I think there’s room to add “Tutorials” and “How-To Guides”.
Was it worth it?
It’s still early but two things have happened:
- The team enjoys making edits to the developer portal.
- The number of support tickets has dropped. The feedback has shifted to the content itself.
That’s a good problem and the next thing to tackle.