SubstackAPI on GitHub

SubstackAPI is open source and available on GitHub.

​
Challenge

Substack does not offer a public API. There are currently two ways to access data from a Substack publication:
  1. Use the Substack publication RSS feed
  2. Use the Substack unofficial/internal API
Both of these options come with their own set of pros and cons. RSS Feed:
  • ✓ Publicly available and stable
  • ✓ Official support from Substack
  • ✗ Limited data access
  • ✗ Slower query performance
Unofficial API:
  • ✓ Rich data access
  • ✓ Fast query performance
  • ✗ No official support
  • ✗ Could break without notice
  • ✗ Restrictive rate limits
Neither option alone provides a complete solution for content creators to access their data reliably and efficiently.

​
Solution

To address these limitations, SubstackAPI implements a hybrid approach:
  1. Primary Data Source: Unofficial API
    • Used to fetch publication data
    • Rate limiting handled through minimizing requests
  2. Fallback Mechanism: RSS Feed
    • Provides backup data access
    • Ensures service continuity if unofficial API changes
  3. Caching Layer: Redis
    • Minimizes calls to the unofficial API
    • Provides fast, low-latency data access
    • Handles large amounts of requests efficiently
This architecture ensures:
  • Reliable and unlimited access to publication data
  • Minimal dependency on the unofficial API
  • Fast response times
  • High availability through fallback mechanisms
The main tradeoff with this solution is a potential delay of up to a few hours for the latest publication updates due to caching. If you are interested in seeing the source code, you can view it on GitHub.