Implementing Backstage Search Collator

How to bring your content to Backstage Search

Backstage logo © Spotify

Backstage is an open platform for building developer portals that includes powerful search functionality. The search can utilize multiple different engines in the backend, such as ElasticSearch, OpenSearch, and Postgres. Items you can search for are provided by components called search collators and in this blog post, we are going to create one. I recommend you first introduce yourself with my previous articles about Backstage.

Basics

Each collator is created by a class that implements the DocumentCollatorFactory interface. These factories can be created directly to the Backstage application or you can create one in a plugin. As you can see from the interface, it’s pretty simple and the most important function is the getCollator which returns a stream containing the search items to be indexed by the backend. Let’s start with a very simple factory that fetches information from some API:

import { getRootLogger } from '@backstage/backend-common';
import { Logger } from 'winston';
import { Readable } from 'stream';

interface ResponseEntity = {
   id: number;
   title: string;
   description: string;
}

export class MyCollatorFactory
  implements DocumentCollatorFactory
{
  type: string;
  visibilityPermission?: Permission;
  private logger: Logger;
  private apiUrl: string;

  private constructor(config: Config) {
    this.logger = getRootLogger();
    this.type = 'my-api';
    this.apiUrl = config.getString('myApi.url');
  }

  static fromConfig(config: Config) {
    return new MyCollatorFactory(config);
  }

  async getCollator(): Promise<Readable> {
    return Readable.from(this.execute());
  }

  private async *execute(): AsyncGenerator<IndexableDocument> {
    this.logger.info(`Starting ${this.type}`);
    const res = await fetch(this.apiUrl, {
      method: 'GET',
      headers: {
        'Accept': 'application/json'
      }
    });
    
    // Handle errors etc.
    const json = (await res.json()) as ResponseEntity[];
    for (const entity in json) {
       const document: IndexableDocument = {
          title: entity.title,
          text: entity.description,
          location: `${this.apiUrl}/${entity.id}`
       };
       yield document;
    }
    
    this.logger.info(`${this.type} DONE!`);
  }…