Price Feed Failures

Lately, I have been having trouble keeping a connection to Oanda's pricefeed. I keep getting disconnected and then eventually my programs tolerance level is exceeded and it times out and dies. If you will recall, last year I migrated everything to the cloud. Sadly when I did that migration I failed to replace my fail-over system which alerted me when failures like this occurred so I could take action. Since my desktop was my trading server until that point, it was simple enough to have alerts be generated on it to awaken me even from sleep if necessary.

Due to programming changes I made in separating out the individual bots from the price feeds, the bots themselves are completely unaffected by the price feed failure, at least from a technical standpoint. From a practical standpoint, they will not be getting any updated pricing information, and will therefore not be able to properly manage trades. I think the next step here is to implement some more automatic failovers to potentially other datafeeds if necessary. I have found a couple potential feeds that can work as a failover in a pinch -- the question is would you rather have data that is potentially 10 pips or more off your brokers feed, or no data at all?

In most cases (depending on the strategy) I think having a general idea of where the price is, even if it's a bit off, is better than sitting blind. If we execute trades on the prices generated from the failover feed, we'll chalk up the execution price against the opening price as slippage anyway.

Automatically failing over, and then rotating back to the real feed at some point will be a technical brain teaser to solve. The most interesting piece will be making the transitions transparent to the strategies using the feed. However, I also need to bulk up my alerting system so I am aware of what's going on as needed. This will likely consist of getting email alerts sent to my phone again. As it stands, if I can get the email, I can login and respond to whatever issue may be occurring. Last but not least, I need to figure out why the feed is failing and if possible fix it.

I started what is now a half finished debugging session almost a week ago. There has been with no movement on my end to finish it. The potential for some better trading should hopefully be motivation enough to take a serious look and overhaul the alerting, failover and reporting systems I'm currently using. The importance of this infrastructure can't be understated. Now to find the time to implement.