No description
- TypeScript 100%
| examples | ||
| src | ||
| tests | ||
| .gitignore | ||
| .prettierignore | ||
| .prettierrc | ||
| package.json | ||
| pnpm-lock.yaml | ||
| pnpm-workspace.yaml | ||
| README.md | ||
| rslib.config.ts | ||
| rstest.config.ts | ||
| tsconfig.json | ||
@tbdhdev/scrapers
TypeScript API wrapper for tbdsux's scraper API projects.
Install
# pnpm
pnpm add @tbdhdev/scrapers
# npm
npm install @tbdhdev/scrapers
# yarn
yarn add @tbdhdev/scrapers
# bun
bun add @tbdhdev/scrapers
Development
# watch building of lib
pnpm run dev
APIs
- SCR-API — Web scraping & screenshots with optional ad-blocking
- Rake — Advanced scraping with multiple backends and Cloudflare bypass
- Fetcher — Simple web page content fetching
SCRApiClient
import { SCRApiClient } from '@tbdhdev/scrapers';
const client = new SCRApiClient();
// const client = new SCRApiClient("https://custom-base-api.tbdh.app/"); // Pass custom base url accordingly
Screenshot
const screenshot = await client.screenshot('https://example.com', {
width: 1920,
height: 1080,
fullPage: false,
imageType: 'png',
waitTimeout: 5,
useAdblock: false,
});
if (!screenshot.success) {
console.error('Error taking screenshot:', screenshot.error);
return;
}
// write to file
const arrBuffer = await screenshot.data.arrayBuffer();
const buffer = Buffer.from(arrBuffer);
writeFileSync('screenshot.png', buffer);
Scrape
const scrape = await client.scrape('https://example.com', {
useAdblock: true,
});
if (!scrape.success) {
console.error('Error scraping:', scrape.error);
return;
}
console.log(`Output of example.com ::>\n\n${scrape.data}`);
RakeClient
import { RakeClient } from '@tbdhdev/scrapers';
const rake = new RakeClient();
// const rake = new RakeClient("https://custom-base-api.tbdh.app/"); // Pass custom base url accordingly
Scrape
const scrape = await rake.scrape('https://example.com', {
response: 'md', // "html" | "md" .. default: "md"
scraper: 'flaresolverr', // "flaresolverr" | "flaresolverr-alt" | "primp" | "requests" | "flare-bypasser"
});
if (!scrape.success) {
console.error('Error scraping:', scrape.error);
return;
}
console.log(`Output of example.com ::>\n\n${scrape.data}`);
Check API Status
const status = await rake.get();
if (status.success) {
console.log('Rake API status:', status.data.message);
console.log('Config:', status.data.config);
}
FetcherClient
import { FetcherClient } from '@tbdhdev/scrapers';
const fetcher = new FetcherClient();
// const fetcher = new FetcherClient("https://custom-base-api.tbdh.app/"); // Pass custom base url accordingly
Scrape
const scrape = await fetcher.scrape('https://example.com');
if (!scrape.success) {
console.error('Error fetching:', scrape.error);
return;
}
console.log(`Output of example.com ::>\n\n${scrape.data}`);
Types
All API clients return a discriminated union type FResponse<T>:
type FResponse<T> =
| { success: true; data: T }
| { success: false; error: string };
Use success to narrow the type and safely access either data or error.
© tbdsux | 2025