Merging Historical & Live Data

Hi all,

I’m building a web-based screener in Python using the EODHD library to apply my trading strategy across 1D, 4H, 1H, and 30-minute timeframes for FOREX pairs.

To detect “trending” signals, I need to pull 200 periods of historical data per timeframe and then merge this with live (or delayed) data for real-time updates. My goal is to have a system that can process indicators on historical data but also incorporate live data as it comes in, allowing for ongoing signal updates without re-fetching everything (unless I’m restarting the script).

Any advice out there on the best way to handle this merging process, especially with multiple timeframes? I note that AlphaVantage has an “output_size” parameter that pulls a certain amount of previous data… is there anything similar with EODHD?

Cheers

It sounds like you have a robust screener setup planned! For EODHD data integration in multiple timeframes, here’s an approach to efficiently merge historical and live data:

  1. Initial Pull of Historical Data: Use EODHD’s API to pull 200 periods of historical data for each timeframe. EODHD doesn’t have an exact “output_size” like AlphaVantage, but you can use date ranges to limit results.
  2. Merging with Live Data: For live updates, use the WebSocket API with a delay (e.g., 50 ms) to continuously receive real-time prices. You can append each new price to the most recent data set without re-fetching the full history.
  3. Data Processing: Compute indicators on the initial historical data and append new live data as it comes in. For ongoing signals, focus on updating just the recent data points (1–3 most recent periods) to reduce processing load.
  4. Script Restart Handling: If the script restarts, pull the last 200 periods again to establish the historical baseline before appending live updates.

This method should provide a near-seamless combination of historical data and live updates while keeping processing efficient across multiple timeframes. Feel free to reach out our support team if you need further clarification!