Production-Grade Token Counter

Analyze:

  • PDF
  • TXT
  • MD
  • DOCX
  • PPTX

Features:

  • Real GPT token counting using tiktoken
  • Streaming architecture
  • Scanned PDF detection
  • Large file support
  • Batch processing

Notes

  • Uses real GPT tokenizer (cl100k_base)
  • Scanned/image-only PDFs are skipped
  • Optimized for very large documents
  • Batch processing supported