Social Media Toolkit

Maturity: alpha. The normalized PostBundle contract is versioned, but public platform pages and endpoints can change without notice.

Social Media Toolkit is an open-source Python SDK, socialkit CLI, and Model Context Protocol (MCP) server for developers and AI agents that need one auditable interface for public social-media extraction. It does not require a browser profile, cookies, a logged-in session, Agent Switch, or the maintainer's machine.

Quick start

Install or update

Prerequisite: Node.js 18+ with npm / npx.

npx -y github:JNHFlow21/social-media-toolkit
socialkit doctor

The installer:

1. reuses uv when available or downloads it from the official Astral installer;
2. installs the Python package in an isolated uv tool environment;
3. exposes socialkit and social-media-toolkit-mcp on the user command path;
4. never reads provider credentials or creates a project .env file.

socialkit doctor returns JSON. A machine with only the free public-reading path configured can report "status": "partial"; use its warnings field to see which optional text or long-recording capabilities are unavailable.

{
  "status": "partial",
  "supported_platforms": ["douyin", "xiaohongshu", "bilibili", "youtube"],
  "local_asr_fallback": false,
  "warnings": ["optional capability setup guidance"]
}

Run the same npx command to update. Uninstall with:

uv tool uninstall social-media-toolkit

git clone https://github.com/JNHFlow21/social-media-toolkit.git
cd social-media-toolkit
uv sync --locked
uv run socialkit doctor

First useful result

Inspect a public share URL without downloading media or starting GetNote / ASR:

socialkit inspect "SHARE_URL"

The command returns a normalized, provenance-bearing bundle:

{
  "schema_version": "1.0",
  "source": {"platform": "youtube", "url": "https://example.invalid/public-post"},
  "post": {"title": "Example public post"},
  "media": {"videos": [], "covers": [], "images": [], "audio": []},
  "content": {},
  "comments": {},
  "provenance": {"routes": ["platform:public"]}
}

The example is synthetic. Real fields depend on what the public source exposes.

What it can do

Platform matrix

Douyin comment sorting applies only to the public sample returned by the source. It is not a platform-wide ranking, and the toolkit does not paginate or backfill missing items.

Text and transcript routes

The normal readable-text route has one deterministic order:

flowchart LR
    U["Public link"] --> G["GetNote original content"]
    G -->|"no usable text"| N["Native subtitle or post body"]
    N -->|"video still has no text"| V["Volcengine cloud ASR"]
    V -->|"failure"| E["Explicit error"]

There is no local Whisper fallback, alternate ASR provider, OCR/Vision fallback, LLM rewriting, browser automation, CDP, Playwright, or cookie-backed extraction.

Timed YouTube mode is a separate evidence route because every segment must point back to the source video:

manual subtitle cues → automatic subtitle cues → timed Volcengine ASR

It writes only the requested MD/SRT/JSON artifacts to an explicit output directory. Temporary media is deleted before the call returns. --force-asr --speaker-info can bypass existing captions and request anonymous speaker clusters such as SPEAKER_01; those labels do not identify real people.

Duration-aware ASR

The standard route uploads one temporary private TOS object, submits a presigned URL, and deletes the object on success or failure.

Optional configuration

Public metadata, public media downloads, supported public comments, and native captions do not require a Volcengine key.

GetNote

npm install -g @getnote/cli
getnote auth login

GetNote manages its own credentials. Its service may require a paid membership. Calling text, a text-enabled capture, or a full-chain smoke test is the execution signal for the documented route and can save the URL to the user's GetNote account.

Volcengine ASR and TOS

Secret names:

VOLCENGINE_ASR_API_KEY
TOS_ACCESS_KEY
TOS_SECRET_KEY

Non-sensitive long-recording settings:

TOS_BUCKET
TOS_REGION
TOS_ENDPOINT
TOS_OBJECT_PREFIX            # optional
TOS_PRESIGN_EXPIRES          # optional

Inject secrets through the operating system, MCP client, or another secret manager. Do not commit them or write them to a project .env file. Non-sensitive TOS settings may instead live at ~/.config/social-media-toolkit/config.json:

{
  "volcengine_tos": {
    "bucket": "example-private-bucket",
    "region": "example-region",
    "endpoint": "https://example-tos-endpoint.invalid",
    "object_prefix": "social-media-toolkit/long-asr"
  }
}

The repository never stores secret values. Agent Switch is an optional maintainer integration, not a package dependency.

CLI examples

# Metadata only: no persistent write, GetNote, or ASR
socialkit inspect "SHARE_URL"

# Readable canonical text; may use GetNote or paid ASR
socialkit text "SHARE_URL"

# Timed YouTube transcript artifacts
socialkit text "YOUTUBE_URL" \
  --timed \
  --output "/absolute/path/to/transcripts" \
  --outputs md,srt,json

# Force timed ASR with anonymous speaker clusters
socialkit text "YOUTUBE_URL" \
  --timed --force-asr --speaker-info \
  --asr-context-file "/absolute/path/to/public-context.json" \
  --output "/absolute/path/to/transcripts" \
  --outputs json,md,srt

# Bounded Douyin public-comment sample
socialkit comments "DOUYIN_URL" --sort likes --limit 20

# Explicit persistent media download
socialkit download "SHARE_URL" \
  --include video,cover,images \
  --output "/absolute/path/to/output"

--speaker-info and --asr-context-file require --force-asr. Context is a bounded public-metadata vocabulary hint, not identity mapping.

Python SDK

from social_media_toolkit import SocialMediaToolkit

toolkit = SocialMediaToolkit()

bundle = toolkit.inspect("SHARE_URL")
text = toolkit.get_text("SHARE_URL")
timed = toolkit.get_text(
    "YOUTUBE_URL",
    timed=True,
    output_dir="/absolute/path/to/transcripts",
    outputs="md,srt,json",
)
comments = toolkit.get_comments("DOUYIN_URL", sort_by="likes", limit=20)

The SDK, CLI, and MCP server use the same SocialMediaToolkit orchestrator and the same versioned result contracts.

MCP server

Start the stdio server:

social-media-toolkit-mcp

Client configuration:

{
  "mcpServers": {
    "social-media-toolkit": {
      "command": "social-media-toolkit-mcp"
    }
  }
}

Do not place secrets in this JSON. Use the client's secret store or secure process environment.

Trust and boundaries

The downloader accepts HTTP(S), rejects local/private literal IPs, bounds download size, sanitizes filenames, and records SHA-256 manifests. See SECURITY.md for private vulnerability reporting.

Compatibility and distribution

• Python 3.10–3.12 are the supported package runtimes.
• Node.js 18+ is required only for the one-command bootstrap.
• macOS, Linux, and Windows are exercised by CI; platform endpoints themselves may still change independently.
• The canonical user distribution is the GitHub-backed npx installer and isolated uv tool environment.
• Versioned source distributions, wheels, npm tarballs, and checksums are attached to GitHub Releases.

Architecture and documentation

• Architecture and side effects
• Capability matrix and limitations
• Release history
• Contributor guide
• Security policy
• Machine-readable project map

Development

uv sync --locked
uv run python -m unittest discover -s tests
uv run python -m compileall social_media_toolkit social_post_extractor_mcp
uv build
npm test
npm pack --dry-run
git diff --check

Tests use synthetic fixtures and must not contain real cookies, tokens, private media, or user content.

Repository activity

Contributing, support, and license

Bug reports and focused feature proposals are welcome through GitHub Issues. Read CONTRIBUTING.md before opening a pull request, and use GitHub's private vulnerability reporting for security issues.

Licensed under the Apache License 2.0.