NestJS

使用 Firecrawl 与 NestJS 构建结构化的网页抓取与搜索服务。

​

前置条件

• NestJS 项目 (@nestjs/cli)
• 一个 Firecrawl API 密钥——免费获取一个

​

安装 SDK

npm install @mendable/firecrawl-js

将你的 API 密钥添加到 .env 文件中：

FIRECRAWL_API_KEY=fc-YOUR-API-KEY

​

创建 Firecrawl 服务

创建 src/firecrawl/firecrawl.service.ts：

import { Injectable } from "@nestjs/common";
import Firecrawl from "@mendable/firecrawl-js";

@Injectable()
export class FirecrawlService {
  private readonly client: Firecrawl;

  constructor() {
    this.client = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });
  }

  async search(query: string, limit = 5) {
    return this.client.search(query, { limit });
  }

  async scrape(url: string) {
    return this.client.scrape(url);
  }

  async interact(url: string, prompts: string[]) {
    const result = await this.client.scrape(url, { formats: ['markdown'] });
    const scrapeId = result.metadata?.scrapeId;

    const responses = [];
    for (const prompt of prompts) {
      const response = await this.client.interact(scrapeId, { prompt });
      responses.push(response);
    }

    await this.client.stopInteraction(scrapeId);
    return responses;
  }
}

​

创建控制器

新建 src/firecrawl/firecrawl.controller.ts：

import { Body, Controller, Post } from "@nestjs/common";
import { FirecrawlService } from "./firecrawl.service";

@Controller("firecrawl")
export class FirecrawlController {
  constructor(private readonly firecrawlService: FirecrawlService) {}

  @Post("search")
  async search(@Body("query") query: string) {
    return this.firecrawlService.search(query);
  }

  @Post("scrape")
  async scrape(@Body("url") url: string) {
    return this.firecrawlService.scrape(url);
  }

  @Post("interact")
  async interact(@Body("url") url: string, @Body("prompts") prompts: string[]) {
    return this.firecrawlService.interact(url, prompts);
  }
}

​

注册模块

创建 src/firecrawl/firecrawl.module.ts：

import { Module } from "@nestjs/common";
import { FirecrawlService } from "./firecrawl.service";
import { FirecrawlController } from "./firecrawl.controller";

@Module({
  providers: [FirecrawlService],
  controllers: [FirecrawlController],
  exports: [FirecrawlService],
})
export class FirecrawlModule {}

在 AppModule 中导入 FirecrawlModule。

​

试一试

curl -X POST http://localhost:3000/firecrawl/search \
  -H "Content-Type: application/json" \
  -d '{"query": "firecrawl web scraping"}'

​

后续步骤

Scrape 文档

所有 scrape 选项，包括 formats、actions 和代理设置

Search 文档

进行网页搜索并获取完整页面内容

Interact 文档

点击、填写表单并提取动态内容

Node SDK 参考

完整的 SDK 参考，涵盖爬取、map、batch scrape 等功能