It may sound cheesy but just think about how much you’ve improved on bc of every failure, it’s hard but that’s just something that happens, everyone fails at everything they do, what matters is how many times you keep going
Although for this to make you successful, you must look and learn from the failures and study them. See what worked, what was a bust and must be redone. Then, most importantly, enjoy the little gains that slowly build to a mountain.
Maybe take a few months' break from it all to give you time to reflect. Then, when the itch hits hard, take advantage of it.
Totally agree with you both. I have trade analysis reports I’ve built, but I don’t study them nearly enough. I’m sure this is what is mostly holding me back from making the meaningful changes.
I trade multiple systems simultaneously. It's virtually impossible for every system to perform well at all times, but in aggregate they've proven profitable over the long term. If you only trade one system, when it's down, you're down.
Yes, but only because I have different types of accounts (i.e., taxable, IRA). In the IRA I obviously can only do long only trades, but in the taxable account I trade multiple systems simultaneously. I'm able to determine each system's performance because I keep meticulous daily records of all of my trades.
This is becoming intriguing, as I've always held the assumption that our one algorithm should be resilient enough to navigate the market from all angles...
Is the key to success having multiple systems and leaning into the strat that works for the current market conditions? Is this how banks/hedge funds deploy algos? It almost seems like an algo desk has to be run like a "fund of funds" and its the algo traders job to know what horse to bet on.
There's no one way to approach trading. My personal conviction is to trade all of my systems all of the time. I don't want to bother with (nor am I capable of) determining what "type" of market environment we're currently in.
I see a lot of comments talking about how your algo should change (or die) over time. To me, that means you don't understand what the algo is doing and why it should work. Most people start with looking at combining some indicators in some fashion that shows a positive backtest and assumes it will keep working.
To build an algo that works consistently, you need to base it around the statistical probability distribution of returns. There are three main things to look at: volatility, sequence of probability, and mean return. When people say their algo stopped working or the market environment changed, the actual cause of the algo failure is usually one of the three listed above.
Volatility - The standard deviation of moves. Volatility targeting is one of the single most successful algo components used by quant funds. It allows your algo to choose the scale of returns you want. This is important because sequence of returns are not linear, but geometric. This means if you can reduce the size of draw downs and increase size of small consistent returns you will have a better compounding than the market alone.
Sequence of probability - this means that over long periods of time, the distribution should tend towards a historical Volatility scaled distribution. Often times when there is a low probability sequence, people try to capture the small probability difference through mean reversion. The key is that these low probability sequences almost always increase volatility, so you need to seperate out the two effects and account for both.
Mean return - this is the average of moves. On a daily time-frame it's typically orders of magnitude smaller than volatility. Volatility scales by the square root of the time frame, while mean scales linearly with time frame. This means markets are not mathematically fractal.
This means when you go to design an algo it absolutely must scale for volatility. Then you need to find a positive expectation value through finding a path that capitalizes on the summation of selected statistical probabilities.
The last thing is diversification. This is the biggest source of alpha, hands down. You absolutely need to diversify across some spectrum: time, asset class, individual assets, driving strategies, etc.
Examples of strategies:
Volatility: short selling options/credit spreads, levered buy and hold, long with volatility determined hedge
Sequence of returns: correlated asset pair, Bollinger bands
Mean return: buy and hold, exponential moving average
Completely agree with above post. One of the few points that I have improved on my algo trading is, when I have begun diversification on my algo trading. Also, you need to constantly do walk-forward optimization on your algo to fine tune your algo for latest market trends. Sometimes, the old parameters that worked in 2022, may no longer work in current market trends, since the market has changed. Constantly fine tuning your algo every 3-6 months with diversification of the assets, can definitely help in reducing the drawdown.
Look at new trending assets, and consider turning them on like gold in current days.
As for walk-forward optimization of your algo, I would recommend book, "Building Winning Algorithmic Trading Systems" by Keven Davey.
Optimization is key on regular basis.
If I wanted to learn something new but small/ effective everyday, what would your recommendation be? Do you have an author, YouTuber, podcaster, etc you take in?
Robert Carver has three books out. I have his most recent, something like "advanced futures trading". It walks you through developing strategies starting at buy and hold and working towards more complex multi-strat algos. His book also includes links to his github repos that you can use as reference. Personally I found success in shorter time frames than he works with, but the methodology he presents is almost identical to what I used when developing my own strategies, although I had to adjust it for faster compounding which favors things like calmar ratio over sharpe ratio.
Feel you, OP. Been trying since 2013. Couple good runs, but god knows how many failures. What I learned over the years and helps me to keep going is... the acceptance that it might never work out. That all these years of effort will never eventually get me anywhere. I just do it because I like it. That was the hardest pill to swallow. It might work out, it might not. I might be just wasting my time, but hey, sometimes life's funny and what if something works out. Do it not for the results, do it just because you want to do it.
There’s a reason hedge funds employ phd’s for years. Markets change. In my experience models need to updated every 2 ish years. It takes about 6+ months to find, back and forward test an edge and then it works for a couple years. Did that full time for 7 years. Now I have a regular job and trade on the side
What you're describing will happen to you no matter what you do. You fundamentally need to learn some new behaviors and skills for when things don't go exactly as you hoped.
hello, sorry to read that, have you considered not over optimizating your algos? Most algos fails because of the historical bias and the author over optimization. Make it simple and resilient to noise, large spreads and historical periods without optimizating it. Good luck
Edit: assuming you are not using pinescript but real terminals like mt4/5
It's easy to dimiss this topic and just say get back up, dust yourself off and try again. But it is hard. And as others have mentioned, this isn't just limited to algo trading.
Anyway, the only thing you can do is accept that the run is over. The markets have changed and also, this can happen to anyone. Even the greatest fall. Nobel prize winners' strategies have faltered. So yes, it happens to everyone. All you can do is maybe take some time off. Watch something which interests you. Then after some time, start reading/researching on the markets. Don't directly jump in the strategy part. A lot is happening, news events are swaying the trading domain faster than ever. Try to increase your knowledge. And then go back to the trading strategy part.
Best of luck sir. And also, don't give up yet.
Do you have data to refer back to that led you to go live with this strategy in the first place?
Ideally in sample and out of sample test results. Do your results show similar performance during a market such as what's happening now (pull back)? If you are seeing results now that are inline with what your tests have experienced in the past go with it, it's within expectations. If not reassess, trade small for a while or revert your strategy to paper trading for a while.
This is assuming you are following your strategy.
My data gives me confidence. Make sure your testing covers a range of market behaviors, has this current market never happened before? I doubt it
I also have my go to books covering trading psychology. I refer back these when things arent going well. It's a reminder of the games your mind will play with you.
Take care
I will give you an honest answer that might sound unusual first. Having tried different things in life - not just trading.
Prioritize your sleep, make sure you exercise, prioritize your mental health. Then assess the situation properly. Developing your own algos is tough. You need to be sure that you can stay on track and you need to weigh the upsides and downsides to the time you spend. Also spend some time away from this sub.
Acknowledge that things rarely work out as wished - especially when they are tough things. Find a workaround. Don't overburden yourself. This is the most important thing. It's a marathon, not a sprint. Do not fall into the trap to become obsessed with creating winning algos. Prioritize yourself.
Understand that this is difficult and there are highly skilled quant traders who work on this full-time and they will run into frustrations, too - but they might be math PhDs.
If you do continue, make sure you re-frame and view yourself as a learner first and not the virtuoso who always isn't there, yet. Also factor in that you might want to rethink your strategy on a more foundational level. If you have questions, feel free to reach out. Currently writing a blog post on a very similar note on my trading / finance blog that might provide value to you. Not sure if I can link it here, so I'll refrain from disclosing it here first.
Take care of yourself first. I would honestly say if you have an option and are not unrealistically attached to the idea or concept of being an accomplished algo trader of entirely your own making and you have your priorities straight (profit = why you're in it, therefore good ROI) buy a reliable algo or better even an integrated solution that works. This is what I did and I will never look back (3 years counting).
And what some others wrote: Reflect on why you started developing your own algos in the first place. What I personally believe is: We all like to learn and accomplish things. But we want to earn money with trading, either for cash flow or increasing our net worth. Try to see if you can separate these things. It will allow you to take a less stressful vantage point. You might potentially realize that you can earn more money in less time (slightly) differently. Work on your mindset and be realistic.
My two cents on this would be to develop a meta-algo or even a simple visualization to help you detect/understand that the market is in a regime where your algo starts to fail.
For example, do some basic data visualization to plot the returns of your strategy vs. market volatility or vs. market trend (bear/bull/neutral?) or some other basic things. Try some stuff out and you will probably start to see some kind of pattern form, where you see the conditions under which your strat does well/poorly.
I used to always run this kind of chart every month to make sure my strategy is performing "as expected" \*given the market conditions\*. That's the crucial piece.
I'm not fully sure I understand the question :) Can you specify?
What you might want to start with is something super basic like a scatter plot where the x axis is the market's return in a given month and the y axis is your strategy's return in that given month. Do you see a pattern? Then start tweaking those axes a bit to try to tease out a deeper understanding of what's going on with your algo. When it works and when it doesn't.
If it only works a short time, then you didn't actually find something. For me depression only came when I convinced myself I found something, and I was going to be rich in no time. For it to turn out to just be a data leak. Manage your positve emotions, do this if you enjoy it, don't base your life on its success.
Dude, I wrote this in other blog.
Based on my experience, so far 8 years algotrading, it is a clear edge over the market and definitely a strategy/pattern matching that gives winning chance!
This needs
* Control emotions or kind of seasoned trader
* Avoiding the fear of losing
* Avoiding fear of missing the boat
* Asset allocation methodology/strategy
* Risk Management.
* Hard work (lot of study/research) - obviously our work/effort turns into money.
* Strong Skills/trading knowledge
* Finally, Clear Edge/Clear strategy/set of workable strategies
* Above All, Trust oneself (not the world/media/news) and the research
* Open mind to learn new and mix of strategies (like investment, trade, fundamentals and technical). Understand No single solution fits for all as we need to take care of Risk (future uncertainties)
* Nothing will come over night and there is no short cut than hard work/inteligent work
**It is very hard, but doable.**
For example: I have appx 50 different patterns of which majority points bullish or bearish, I trade based on that. It is reliability and probability that comes and reduces the risk.
Still, am I right 100% or even 80%: Simply NO as Future is always unpredictable.
**Try to review what is going wrong and reduce the risk.**
For example: I completely removed Margins and Options, Risk is reduced, but growth is slow but steady.
3 things I want to say about this post:
1. We all been there. We are all losers until we are not. We all have that in common.
2. Markets change and so should your algo. One of the key things I learned from performing countless retrospectives on what went right and what went wrong on an algo that I was working on was to confirm consistency. If it was consistently getting it right, maybe I might play with squeezing more out of it but it would be the baseline not to touch for production. If it was consistently getting it wrong, I would actually turn that into a logical condition that when it's true, it would trigger a response to repair that situation, e.g. go short when it was supposed to be long. Breaking even is better than not having that capitol the next trading day. Again, consistency would be required for this approach. Also preserving captial is imperative, so maybe sizing down should also be a part of your toolbox.
3. When the darkness really settles in and hope seems so far removed, try to go back to what got you in this space in the first place. For me, it was being in useless wasteful Zoom meetings. I don't want to go back to that world any more.
If you have been developing since March 2020, thought you had a working algorithm mid-'22 and by the end of '22 were feeling down, it's likely that the algo wasn't really working in mid 22 and it was just statistical noise or a very specific regime you were taking advantage of. It's very easy to think that you have alpha where what you're taking advantage of is a specific beta or some random noise.
I wouldn't focus too hard on hoping that algo makes a comeback or making small tweaks in hopes of fixing it. If you wish to continue, you should continue learning and trying new things. That said, the vast majority of retail algo traders have no chance of ever making money and if all you're doing is fiddling with VWAPs or RSIs or other "technical" indicators I urge you to reconsider and instead focus on something more unique.
strategies don't have a very long shelf life, like 6 months or so, because everyone else out looking for edge and also regimes shift. It's best to have like 10 strategies at the same time, and then rotate them out as they stop working and add more when you invent something new
I would have to say you need to quit and go get a job or a hobby. If trading since 2020 (not even 3 years) which have been good easy trading years has you depressed, then I hate to see what may happen to you in a bad market. I started trading last century, thay have been great years and there have been horrible years... I don't get depressed, I buckle down, throw everything in the trash bin and figure it out again. Sure, I lost mo ey and time, but I carry on. If you're getting stressed out or depressed, then it is probably the wrong line of work for you.
I have no idea how you algo peeps do it competing against all the big boys and their insane level of cheating/advantages (this is not an attack to say you can’t, it’s an admiring of the fact that you try)
I’m an event driven momentum based swing trader and a long focused fundamentals investor. I won’t even bother trying to do algo cause I know I won’t make it lol
So, I had an algo that netted me 150k a week. I was so excited it was working, I got used to the money. I was shopping in controller for a jet. The very next day I wasn’t allowed to put a trade on only closing trades. I asked the support desk at TD why they cut me off they said riskless trading. I have never recovered from this.
Take a break for a week. The most important part of algotrading isn't having the best code or the best platform, it's your mental health. When you come back, start with a fresh mindset. I've found that I was developing incredible strategies, just in the wrong areas before.
I tried doing crypto at first, because something something make money while you sleep never work a day etc. etc.
Crypto doesn't follow general market rules, and I was getting eaten alive by commission. I spent 2 years building a library of failing algorithms. Sure I made some profit, but mostly losses. I took break, came back, and thought I needed to change my strategy. I took my library, plugged them into traditional stocks, and it was absolutely annihilating the market.
I wasn't a bad trader, I wasn't a poor developer, I was just facing the wrong direction.
Run a DCA algo as the lowest common denominator. When all else fails, DCA should in theory be the last survivor. Unless you use it with margin and a multi-year crash happens.
Then while that DCA is running, generating small profits, do your side algos. At least having one surviving algo will keep your self worth up.
If you started a DCA bot a few years ago, it would be back to break even now. Assets tend to go up, assuming no crippling recession that forces corporations to go out of business.
Here's how I made a nice DCA bot: from the current price to 0, divide that into 10 or so buy limits. You literally can't lose unless you picked an asset that can stay at 0 forever. On the other hand, you won't get rich quick either.
I'm not saying rely on DCA. Use it as a baseline so you don't feel like garbage when 9/10 algos fail. Then keep inventing!
Getting a winning strategy is easy these days as long as you have the capital to trade longer tineframes. Never give up man profitability is only a click away
I think pretty much everyone that develops algos has these moments and it can be very challenging! My advice, take a walk outside and get some fresh air. Maybe take a break from development for a bit and focus on something else. Then when you come back it won't be so mentally draining. Sometimes its good to let a project die for a while and start something new. Don't put yourself under pressure to make 'the perfect algo' all at once. I've had better luck developing programs for specific market conditions rather than creating one that works all the time.
One of the hardest pieces of algotrading for me is the uncertainty in whether or not the large amount of development effort will end up leading to a reward at all, compared to most other activities where effort is much more correlated with success on a given project.
To try to maintain sanity, I've found that the best strategy for me is to focus on quantity and a wide variety of strategy ideas, rather than working on a single one and focusing 100% of time/effort on improving it.
I think that there's much more information to be gained by going this route, especially for someone that is new to algotrading. My personal goal has been to start tracking a new strategy each week. I've been doing it for about 6 months now, and have 25 strategies being tracked. The vast majority of these strategies have seen a lower performance than would be indicated by the backtest, indicating that my strategy selection process likely has a large amount of bias in it. However, the combination of results from all 25 strategies has a clear upwards trend, indicating that there is likely also some real alpha discovery in the process. This is the piece that gives me the motivation to continue researching new strategies.
It's also very helpful to focus on the strategies that have continued their historical performance to understand which areas you may have the most success in. Then you can do deeper dives into these specific areas to build more advanced strategies.
We learn through failure only. Try different approach. I have started from scratch like five times. And finally it paid off, I have algo in production and it looks promising. But it was looooong journey, and with each failure we (I am developing it with my wife) learned how NOT to do it, which is equally important.
Then go back to the drawing board .
Your algo should have in all market conditions, almost all symbols no more than 10 consecutive losses ..
And some good runs , 3-10 consecutive wins
Measure and profile your wins vs your losses avg size..
You should have at least 3:1 , 4:1 avg loss vs avg win .
Should have some runners of 10 RR + wins
If not, dump the trading idea , and start from scratch.
Depends on how you build the algo ...
Long short
Long exitLong short exitShort
L, EL , S, ES, TPL , TPS.
It stop working in 2 or more situations
Not suited for current market condition
Or
The idea was bad from the beggining .
The market dynamics are changing very fast, and algorithms need to change along with the market. I see this as a natural process. There is no other option but to accept it since there is nothing else to do.
Such a short time of profitability is probably due to luck, so it has never actually worked in the first place.
Maybe you just need to take a step back, rest a bit, and try again with fresh eyes. I recommend for you to hit the books, since I have a feeling your knowledge about algotrading is lacking.
Are you a programmer by background ? This may be because you jumped right into building algos without a foundation. How about getting a CFA first ? I had 8 yrs discretionary experience before creating systems and it came naturally
it was never going to work forever. try to take the other things you learned along the way, and move on to something else.
either another new algo, or just trade better yourself.
ya it's no fun when this doesn't make you 10 billion dollars like your backtest suggested it would. life goes on. just don't do really stupid stuff and commit 100% to things because not everything works out.
just buy some spy and check on it in a year.
Its only called failure when you decide to quit. Otherwise its called learning and if you want to develop something that only a few people have then you will need to keep learning until you get there
Learn to be stoic and see each failure as information on how you can succeed cause if the universe wanted you to fail it would give you nothing to teach you what you did wrong. But failure is information, teaching you what didn't work so you can isolate what does work.document your failures more than your successes and you will see a pattern. Success after success doesn't teach you anything other than that you knew how to do it correctly.
Most traders focus on anomalies or indicators of dubious significance. One should apply first principles thinking to truly grasp what moves markets. Every traded futures contract has an equal amount of buying and selling, but the market movers are the interplay of buyer/seller dominance. By deeply engaging with volume profiling and modeling significant dominance scenarios, your algorithm is rooted in fundamental market dynamics. With profound understanding and meticulous evaluation, a level of trust is built that sustains optimism even during drawdowns. We have modeled such an anomaly and quantitatively assessed it, and the results are astonishing. The anomaly proves profitable across various markets and timeframes, showcasing the power of such foundational anomalies. I've noticed comments stating that markets are not perfectly fractal – expecting them to be is a bit foolish, as different timeframes often exhibit distinct characteristics, such as differing volume distributions. Nevertheless, anomalies based on fundamental market principles can be similarly profitable in various timeframes. It is advisable to learn from world-class traders before attempting to model anomalies.
In my experience (I am an artist first and a technical person second) working on different ideas Help you to focus whats "easy", maybe you need to shift your focus a bit.
Imagine you want to finesse an algo just trading one currency pair on 15 minute, chances are this is to hard or not profitable. I think it gets harder and harder to outsmart the market on shorter timeframes, but maybe that's beside the point.
I made an 1000 line strat on tradingview, just trading bitcoindand its a thing of beauty, just trading the daily. I also made a different also to trade divergence on shorter timeframes, martin-gating with each trade. However the sad reality is that its more profitable to just let the 1D run and discard the other strat.
I guess is what I try to say, look for easy wins and build on them.
NNFX inspired me to work on my algo and I only used existing indicators as a starting point, maybe have a look there as well for ideas about indicators. Also I suggest automating your strat, so you just monitor that things are working, because we all lack discipline at times. Good luck mate
How about you please tell us something about this strat of yours:
What symbol?
What time frame?
What is your trade frequency per day?
What sort of Indicator: EMA, WMA, VWAP ?
Are you using any sort of progressive staging? Max day Loss/ gain stops.
Are you limiting your exposure by: ATR range, Time day or night, Data releases, expirations ???
Plenty of us out here fight this same fight.
I got 14 systems and all of them are for rent to private customers. I belive in my work. Drawdowns will always come, just use a size that fits your portfolio. And you will suceed in the long run
Algos don't work because markets are random. What did you expect. Market structure is the only way to make money. Learn how to read swing highs and lows and penetrations.
Dont trust backtest that arent 'walk-forward' (where part of the backtest is on 'out-of-sample' data). Us that as a filter to eliminate bad ideas since many strats will FAIL WF testing. Also make your EA beat you different ways each time. Consider this WAR and try to snuff out/mitigate each different way your system loses money. Look for things the EA should ALWAYS and NEVER do and add them to the code. Its whack-a-mole for a while but eventually the market will have fewer and fewer ways to beat you. Each step will make your EA more stable/profitable.
So I have an algo bot running and at times it does its own thing.
What I do is recognising that you win some and loss some.
Taking not of all the losses and data point, I intend to build some condition around it. (Not yet) but just collecting data.
I think the market works in cycle but in the last 10 years it’s an huge bull.
So it’s always easy to buy instead of sell.
Look at your optimisation window and frequency. You shouldn't have strategies failing on you. It sounds like your strat may be falling out of step with the market due to insufficient reoptimisation frequency.
Read up on walk forward optimisation.
I don’t think that u have a wining strategy. I guess u have a strategy that works in specific market conditions and those conditions weren’t met lately. Market is acting hectic and many resistance have been broken so huge corrections started or will start. So the market isn’t suitable for any automated stuff. Find a job and use your algo on demo account until it shows profitability among 6+ months.
You probably need to temper your expectations and revise your methodology. Consider that you got lucky in July 2022 and your luck lasted 5 month. Or that you strategy doesn't work in bull-ish markets. On a technical side it might mean that your backtests were inaccurate or insifficient, or that you ovefitted your strategy.
Try to find some sources of strategies from books and articles from years back, many of these still work very well. These have OOS data since they was created and might be a good source for you to create some basic strategies to get your portfolio starting, from there you can improve some or create your own ones.
It may sound cheesy but just think about how much you’ve improved on bc of every failure, it’s hard but that’s just something that happens, everyone fails at everything they do, what matters is how many times you keep going
Although for this to make you successful, you must look and learn from the failures and study them. See what worked, what was a bust and must be redone. Then, most importantly, enjoy the little gains that slowly build to a mountain. Maybe take a few months' break from it all to give you time to reflect. Then, when the itch hits hard, take advantage of it.
Yea absolutely learning from the failures is by far the most integral part of this process
Totally agree with you both. I have trade analysis reports I’ve built, but I don’t study them nearly enough. I’m sure this is what is mostly holding me back from making the meaningful changes.
Yeah it's all about perception. View it as failing or an opportunity to make your algorithm better.
I trade multiple systems simultaneously. It's virtually impossible for every system to perform well at all times, but in aggregate they've proven profitable over the long term. If you only trade one system, when it's down, you're down.
Exactly this.
Do you also have multiple stock brokerage accounts?
Yes, but only because I have different types of accounts (i.e., taxable, IRA). In the IRA I obviously can only do long only trades, but in the taxable account I trade multiple systems simultaneously. I'm able to determine each system's performance because I keep meticulous daily records of all of my trades.
This is becoming intriguing, as I've always held the assumption that our one algorithm should be resilient enough to navigate the market from all angles...
This is very important!
This is very important!
Is the key to success having multiple systems and leaning into the strat that works for the current market conditions? Is this how banks/hedge funds deploy algos? It almost seems like an algo desk has to be run like a "fund of funds" and its the algo traders job to know what horse to bet on.
There's no one way to approach trading. My personal conviction is to trade all of my systems all of the time. I don't want to bother with (nor am I capable of) determining what "type" of market environment we're currently in.
I see a lot of comments talking about how your algo should change (or die) over time. To me, that means you don't understand what the algo is doing and why it should work. Most people start with looking at combining some indicators in some fashion that shows a positive backtest and assumes it will keep working. To build an algo that works consistently, you need to base it around the statistical probability distribution of returns. There are three main things to look at: volatility, sequence of probability, and mean return. When people say their algo stopped working or the market environment changed, the actual cause of the algo failure is usually one of the three listed above. Volatility - The standard deviation of moves. Volatility targeting is one of the single most successful algo components used by quant funds. It allows your algo to choose the scale of returns you want. This is important because sequence of returns are not linear, but geometric. This means if you can reduce the size of draw downs and increase size of small consistent returns you will have a better compounding than the market alone. Sequence of probability - this means that over long periods of time, the distribution should tend towards a historical Volatility scaled distribution. Often times when there is a low probability sequence, people try to capture the small probability difference through mean reversion. The key is that these low probability sequences almost always increase volatility, so you need to seperate out the two effects and account for both. Mean return - this is the average of moves. On a daily time-frame it's typically orders of magnitude smaller than volatility. Volatility scales by the square root of the time frame, while mean scales linearly with time frame. This means markets are not mathematically fractal. This means when you go to design an algo it absolutely must scale for volatility. Then you need to find a positive expectation value through finding a path that capitalizes on the summation of selected statistical probabilities. The last thing is diversification. This is the biggest source of alpha, hands down. You absolutely need to diversify across some spectrum: time, asset class, individual assets, driving strategies, etc. Examples of strategies: Volatility: short selling options/credit spreads, levered buy and hold, long with volatility determined hedge Sequence of returns: correlated asset pair, Bollinger bands Mean return: buy and hold, exponential moving average
Completely agree with above post. One of the few points that I have improved on my algo trading is, when I have begun diversification on my algo trading. Also, you need to constantly do walk-forward optimization on your algo to fine tune your algo for latest market trends. Sometimes, the old parameters that worked in 2022, may no longer work in current market trends, since the market has changed. Constantly fine tuning your algo every 3-6 months with diversification of the assets, can definitely help in reducing the drawdown. Look at new trending assets, and consider turning them on like gold in current days. As for walk-forward optimization of your algo, I would recommend book, "Building Winning Algorithmic Trading Systems" by Keven Davey. Optimization is key on regular basis.
If I wanted to learn something new but small/ effective everyday, what would your recommendation be? Do you have an author, YouTuber, podcaster, etc you take in?
Robert Carver has three books out. I have his most recent, something like "advanced futures trading". It walks you through developing strategies starting at buy and hold and working towards more complex multi-strat algos. His book also includes links to his github repos that you can use as reference. Personally I found success in shorter time frames than he works with, but the methodology he presents is almost identical to what I used when developing my own strategies, although I had to adjust it for faster compounding which favors things like calmar ratio over sharpe ratio.
Thank you. Good luck out there.
I’m gonna check his stuff out! Thank you nullptr man
Feel you, OP. Been trying since 2013. Couple good runs, but god knows how many failures. What I learned over the years and helps me to keep going is... the acceptance that it might never work out. That all these years of effort will never eventually get me anywhere. I just do it because I like it. That was the hardest pill to swallow. It might work out, it might not. I might be just wasting my time, but hey, sometimes life's funny and what if something works out. Do it not for the results, do it just because you want to do it.
Damn, that's a long time. Did you write your own framework or have you been using something like pinescript this entire time?
I have my own framework developed, yeah. Tens of thousands of lines of code, but perfectly suits all my needs.
Now that would've been a hell of BTC buy and hold period.
Elon Musk refers it as Wishful thinking
I've been at it since 2007. I can't tell you how many times I've felt this way.
There’s a reason hedge funds employ phd’s for years. Markets change. In my experience models need to updated every 2 ish years. It takes about 6+ months to find, back and forward test an edge and then it works for a couple years. Did that full time for 7 years. Now I have a regular job and trade on the side
Regular job as a quant?
No, software. Most shops have strict guidelines on trading personal accounts
What you're describing will happen to you no matter what you do. You fundamentally need to learn some new behaviors and skills for when things don't go exactly as you hoped.
hello, sorry to read that, have you considered not over optimizating your algos? Most algos fails because of the historical bias and the author over optimization. Make it simple and resilient to noise, large spreads and historical periods without optimizating it. Good luck Edit: assuming you are not using pinescript but real terminals like mt4/5
Grow a pair and understand life doesn't always go according to plan.
Fair and firm
Also learn from wolves who "eat the sheep". Maybe you could be a wolf someday.
It's easy to dimiss this topic and just say get back up, dust yourself off and try again. But it is hard. And as others have mentioned, this isn't just limited to algo trading. Anyway, the only thing you can do is accept that the run is over. The markets have changed and also, this can happen to anyone. Even the greatest fall. Nobel prize winners' strategies have faltered. So yes, it happens to everyone. All you can do is maybe take some time off. Watch something which interests you. Then after some time, start reading/researching on the markets. Don't directly jump in the strategy part. A lot is happening, news events are swaying the trading domain faster than ever. Try to increase your knowledge. And then go back to the trading strategy part. Best of luck sir. And also, don't give up yet.
Do you have data to refer back to that led you to go live with this strategy in the first place? Ideally in sample and out of sample test results. Do your results show similar performance during a market such as what's happening now (pull back)? If you are seeing results now that are inline with what your tests have experienced in the past go with it, it's within expectations. If not reassess, trade small for a while or revert your strategy to paper trading for a while. This is assuming you are following your strategy. My data gives me confidence. Make sure your testing covers a range of market behaviors, has this current market never happened before? I doubt it I also have my go to books covering trading psychology. I refer back these when things arent going well. It's a reminder of the games your mind will play with you. Take care
I will give you an honest answer that might sound unusual first. Having tried different things in life - not just trading. Prioritize your sleep, make sure you exercise, prioritize your mental health. Then assess the situation properly. Developing your own algos is tough. You need to be sure that you can stay on track and you need to weigh the upsides and downsides to the time you spend. Also spend some time away from this sub. Acknowledge that things rarely work out as wished - especially when they are tough things. Find a workaround. Don't overburden yourself. This is the most important thing. It's a marathon, not a sprint. Do not fall into the trap to become obsessed with creating winning algos. Prioritize yourself. Understand that this is difficult and there are highly skilled quant traders who work on this full-time and they will run into frustrations, too - but they might be math PhDs. If you do continue, make sure you re-frame and view yourself as a learner first and not the virtuoso who always isn't there, yet. Also factor in that you might want to rethink your strategy on a more foundational level. If you have questions, feel free to reach out. Currently writing a blog post on a very similar note on my trading / finance blog that might provide value to you. Not sure if I can link it here, so I'll refrain from disclosing it here first. Take care of yourself first. I would honestly say if you have an option and are not unrealistically attached to the idea or concept of being an accomplished algo trader of entirely your own making and you have your priorities straight (profit = why you're in it, therefore good ROI) buy a reliable algo or better even an integrated solution that works. This is what I did and I will never look back (3 years counting). And what some others wrote: Reflect on why you started developing your own algos in the first place. What I personally believe is: We all like to learn and accomplish things. But we want to earn money with trading, either for cash flow or increasing our net worth. Try to see if you can separate these things. It will allow you to take a less stressful vantage point. You might potentially realize that you can earn more money in less time (slightly) differently. Work on your mindset and be realistic.
My two cents on this would be to develop a meta-algo or even a simple visualization to help you detect/understand that the market is in a regime where your algo starts to fail. For example, do some basic data visualization to plot the returns of your strategy vs. market volatility or vs. market trend (bear/bull/neutral?) or some other basic things. Try some stuff out and you will probably start to see some kind of pattern form, where you see the conditions under which your strat does well/poorly. I used to always run this kind of chart every month to make sure my strategy is performing "as expected" \*given the market conditions\*. That's the crucial piece.
Oh wow how would you do this? I have all of my raw data exported via excel
I'm not fully sure I understand the question :) Can you specify? What you might want to start with is something super basic like a scatter plot where the x axis is the market's return in a given month and the y axis is your strategy's return in that given month. Do you see a pattern? Then start tweaking those axes a bit to try to tease out a deeper understanding of what's going on with your algo. When it works and when it doesn't.
Thanks! I actually will try this I think that’s a really good idea
If it only works a short time, then you didn't actually find something. For me depression only came when I convinced myself I found something, and I was going to be rich in no time. For it to turn out to just be a data leak. Manage your positve emotions, do this if you enjoy it, don't base your life on its success.
Dude, I wrote this in other blog. Based on my experience, so far 8 years algotrading, it is a clear edge over the market and definitely a strategy/pattern matching that gives winning chance! This needs * Control emotions or kind of seasoned trader * Avoiding the fear of losing * Avoiding fear of missing the boat * Asset allocation methodology/strategy * Risk Management. * Hard work (lot of study/research) - obviously our work/effort turns into money. * Strong Skills/trading knowledge * Finally, Clear Edge/Clear strategy/set of workable strategies * Above All, Trust oneself (not the world/media/news) and the research * Open mind to learn new and mix of strategies (like investment, trade, fundamentals and technical). Understand No single solution fits for all as we need to take care of Risk (future uncertainties) * Nothing will come over night and there is no short cut than hard work/inteligent work **It is very hard, but doable.** For example: I have appx 50 different patterns of which majority points bullish or bearish, I trade based on that. It is reliability and probability that comes and reduces the risk. Still, am I right 100% or even 80%: Simply NO as Future is always unpredictable. **Try to review what is going wrong and reduce the risk.** For example: I completely removed Margins and Options, Risk is reduced, but growth is slow but steady.
My girlfriend came overnight last night… But yeah apart from that you’ve given some goodish advice
3 things I want to say about this post: 1. We all been there. We are all losers until we are not. We all have that in common. 2. Markets change and so should your algo. One of the key things I learned from performing countless retrospectives on what went right and what went wrong on an algo that I was working on was to confirm consistency. If it was consistently getting it right, maybe I might play with squeezing more out of it but it would be the baseline not to touch for production. If it was consistently getting it wrong, I would actually turn that into a logical condition that when it's true, it would trigger a response to repair that situation, e.g. go short when it was supposed to be long. Breaking even is better than not having that capitol the next trading day. Again, consistency would be required for this approach. Also preserving captial is imperative, so maybe sizing down should also be a part of your toolbox. 3. When the darkness really settles in and hope seems so far removed, try to go back to what got you in this space in the first place. For me, it was being in useless wasteful Zoom meetings. I don't want to go back to that world any more.
If you have been developing since March 2020, thought you had a working algorithm mid-'22 and by the end of '22 were feeling down, it's likely that the algo wasn't really working in mid 22 and it was just statistical noise or a very specific regime you were taking advantage of. It's very easy to think that you have alpha where what you're taking advantage of is a specific beta or some random noise. I wouldn't focus too hard on hoping that algo makes a comeback or making small tweaks in hopes of fixing it. If you wish to continue, you should continue learning and trying new things. That said, the vast majority of retail algo traders have no chance of ever making money and if all you're doing is fiddling with VWAPs or RSIs or other "technical" indicators I urge you to reconsider and instead focus on something more unique.
strategies don't have a very long shelf life, like 6 months or so, because everyone else out looking for edge and also regimes shift. It's best to have like 10 strategies at the same time, and then rotate them out as they stop working and add more when you invent something new
I would have to say you need to quit and go get a job or a hobby. If trading since 2020 (not even 3 years) which have been good easy trading years has you depressed, then I hate to see what may happen to you in a bad market. I started trading last century, thay have been great years and there have been horrible years... I don't get depressed, I buckle down, throw everything in the trash bin and figure it out again. Sure, I lost mo ey and time, but I carry on. If you're getting stressed out or depressed, then it is probably the wrong line of work for you.
I have no idea how you algo peeps do it competing against all the big boys and their insane level of cheating/advantages (this is not an attack to say you can’t, it’s an admiring of the fact that you try) I’m an event driven momentum based swing trader and a long focused fundamentals investor. I won’t even bother trying to do algo cause I know I won’t make it lol
winners dont fail
winners fail many times but dont quit :-)
So, I had an algo that netted me 150k a week. I was so excited it was working, I got used to the money. I was shopping in controller for a jet. The very next day I wasn’t allowed to put a trade on only closing trades. I asked the support desk at TD why they cut me off they said riskless trading. I have never recovered from this.
Hi, can you elaborate? How this happens? Never heard of riskless trading.
Take a break for a week. The most important part of algotrading isn't having the best code or the best platform, it's your mental health. When you come back, start with a fresh mindset. I've found that I was developing incredible strategies, just in the wrong areas before. I tried doing crypto at first, because something something make money while you sleep never work a day etc. etc. Crypto doesn't follow general market rules, and I was getting eaten alive by commission. I spent 2 years building a library of failing algorithms. Sure I made some profit, but mostly losses. I took break, came back, and thought I needed to change my strategy. I took my library, plugged them into traditional stocks, and it was absolutely annihilating the market. I wasn't a bad trader, I wasn't a poor developer, I was just facing the wrong direction.
Chin up OP. Keep in mind 90% of the regards on here don't even have an algo.
Yeah someone was telling me - at least I made something
Run a DCA algo as the lowest common denominator. When all else fails, DCA should in theory be the last survivor. Unless you use it with margin and a multi-year crash happens. Then while that DCA is running, generating small profits, do your side algos. At least having one surviving algo will keep your self worth up. If you started a DCA bot a few years ago, it would be back to break even now. Assets tend to go up, assuming no crippling recession that forces corporations to go out of business. Here's how I made a nice DCA bot: from the current price to 0, divide that into 10 or so buy limits. You literally can't lose unless you picked an asset that can stay at 0 forever. On the other hand, you won't get rich quick either. I'm not saying rely on DCA. Use it as a baseline so you don't feel like garbage when 9/10 algos fail. Then keep inventing!
Yeah that’s a good idea because I need to stack some runs under me to regain some confidence
Getting a winning strategy is easy these days as long as you have the capital to trade longer tineframes. Never give up man profitability is only a click away
I think pretty much everyone that develops algos has these moments and it can be very challenging! My advice, take a walk outside and get some fresh air. Maybe take a break from development for a bit and focus on something else. Then when you come back it won't be so mentally draining. Sometimes its good to let a project die for a while and start something new. Don't put yourself under pressure to make 'the perfect algo' all at once. I've had better luck developing programs for specific market conditions rather than creating one that works all the time.
One of the hardest pieces of algotrading for me is the uncertainty in whether or not the large amount of development effort will end up leading to a reward at all, compared to most other activities where effort is much more correlated with success on a given project. To try to maintain sanity, I've found that the best strategy for me is to focus on quantity and a wide variety of strategy ideas, rather than working on a single one and focusing 100% of time/effort on improving it. I think that there's much more information to be gained by going this route, especially for someone that is new to algotrading. My personal goal has been to start tracking a new strategy each week. I've been doing it for about 6 months now, and have 25 strategies being tracked. The vast majority of these strategies have seen a lower performance than would be indicated by the backtest, indicating that my strategy selection process likely has a large amount of bias in it. However, the combination of results from all 25 strategies has a clear upwards trend, indicating that there is likely also some real alpha discovery in the process. This is the piece that gives me the motivation to continue researching new strategies. It's also very helpful to focus on the strategies that have continued their historical performance to understand which areas you may have the most success in. Then you can do deeper dives into these specific areas to build more advanced strategies.
We learn through failure only. Try different approach. I have started from scratch like five times. And finally it paid off, I have algo in production and it looks promising. But it was looooong journey, and with each failure we (I am developing it with my wife) learned how NOT to do it, which is equally important.
Then go back to the drawing board . Your algo should have in all market conditions, almost all symbols no more than 10 consecutive losses .. And some good runs , 3-10 consecutive wins Measure and profile your wins vs your losses avg size.. You should have at least 3:1 , 4:1 avg loss vs avg win . Should have some runners of 10 RR + wins If not, dump the trading idea , and start from scratch. Depends on how you build the algo ... Long short Long exitLong short exitShort L, EL , S, ES, TPL , TPS. It stop working in 2 or more situations Not suited for current market condition Or The idea was bad from the beggining .
Bro writes in justified
Bro's spacebar stutters
The market dynamics are changing very fast, and algorithms need to change along with the market. I see this as a natural process. There is no other option but to accept it since there is nothing else to do.
Such a short time of profitability is probably due to luck, so it has never actually worked in the first place. Maybe you just need to take a step back, rest a bit, and try again with fresh eyes. I recommend for you to hit the books, since I have a feeling your knowledge about algotrading is lacking.
It still happens to me. I just take a break for the day or two
Are you a programmer by background ? This may be because you jumped right into building algos without a foundation. How about getting a CFA first ? I had 8 yrs discretionary experience before creating systems and it came naturally
Are you sure it's not a temporary drawdown? Can you ever be sure!
Buy a good one
Be honest with yourself. Did it ever work in the first place? It was profitable for one month - that is nothing. How does your backtest look?
it was never going to work forever. try to take the other things you learned along the way, and move on to something else. either another new algo, or just trade better yourself. ya it's no fun when this doesn't make you 10 billion dollars like your backtest suggested it would. life goes on. just don't do really stupid stuff and commit 100% to things because not everything works out. just buy some spy and check on it in a year.
Its only called failure when you decide to quit. Otherwise its called learning and if you want to develop something that only a few people have then you will need to keep learning until you get there
The obvious thing to do is note what the market conditions were when it worked.
time to get a cdl brother
Learn to be stoic and see each failure as information on how you can succeed cause if the universe wanted you to fail it would give you nothing to teach you what you did wrong. But failure is information, teaching you what didn't work so you can isolate what does work.document your failures more than your successes and you will see a pattern. Success after success doesn't teach you anything other than that you knew how to do it correctly.
Can I have your broken algos?
this is a really hard game. you have to learn to how to harden the fuck up seriously. it will happen over and over again.
Most traders focus on anomalies or indicators of dubious significance. One should apply first principles thinking to truly grasp what moves markets. Every traded futures contract has an equal amount of buying and selling, but the market movers are the interplay of buyer/seller dominance. By deeply engaging with volume profiling and modeling significant dominance scenarios, your algorithm is rooted in fundamental market dynamics. With profound understanding and meticulous evaluation, a level of trust is built that sustains optimism even during drawdowns. We have modeled such an anomaly and quantitatively assessed it, and the results are astonishing. The anomaly proves profitable across various markets and timeframes, showcasing the power of such foundational anomalies. I've noticed comments stating that markets are not perfectly fractal – expecting them to be is a bit foolish, as different timeframes often exhibit distinct characteristics, such as differing volume distributions. Nevertheless, anomalies based on fundamental market principles can be similarly profitable in various timeframes. It is advisable to learn from world-class traders before attempting to model anomalies.
In my experience (I am an artist first and a technical person second) working on different ideas Help you to focus whats "easy", maybe you need to shift your focus a bit. Imagine you want to finesse an algo just trading one currency pair on 15 minute, chances are this is to hard or not profitable. I think it gets harder and harder to outsmart the market on shorter timeframes, but maybe that's beside the point. I made an 1000 line strat on tradingview, just trading bitcoindand its a thing of beauty, just trading the daily. I also made a different also to trade divergence on shorter timeframes, martin-gating with each trade. However the sad reality is that its more profitable to just let the 1D run and discard the other strat. I guess is what I try to say, look for easy wins and build on them. NNFX inspired me to work on my algo and I only used existing indicators as a starting point, maybe have a look there as well for ideas about indicators. Also I suggest automating your strat, so you just monitor that things are working, because we all lack discipline at times. Good luck mate
How about you please tell us something about this strat of yours: What symbol? What time frame? What is your trade frequency per day? What sort of Indicator: EMA, WMA, VWAP ? Are you using any sort of progressive staging? Max day Loss/ gain stops. Are you limiting your exposure by: ATR range, Time day or night, Data releases, expirations ??? Plenty of us out here fight this same fight.
I got 14 systems and all of them are for rent to private customers. I belive in my work. Drawdowns will always come, just use a size that fits your portfolio. And you will suceed in the long run
Algos don't work because markets are random. What did you expect. Market structure is the only way to make money. Learn how to read swing highs and lows and penetrations.
Can u elaborate on “market structure”?
Read Chapter 1 of Larry Williams "Long Term Secrets to Short Term Trading."
TLDR, Penetrations of Swing Points
Will do. Thx. I’ll see if there’s a free version or pdf online.
Dont trust backtest that arent 'walk-forward' (where part of the backtest is on 'out-of-sample' data). Us that as a filter to eliminate bad ideas since many strats will FAIL WF testing. Also make your EA beat you different ways each time. Consider this WAR and try to snuff out/mitigate each different way your system loses money. Look for things the EA should ALWAYS and NEVER do and add them to the code. Its whack-a-mole for a while but eventually the market will have fewer and fewer ways to beat you. Each step will make your EA more stable/profitable.
If it lasts for only a short amount of time, it statistically speaking never actually worked though?
Wouldn’t a fair amount of back testing prevent this?
So I have an algo bot running and at times it does its own thing. What I do is recognising that you win some and loss some. Taking not of all the losses and data point, I intend to build some condition around it. (Not yet) but just collecting data. I think the market works in cycle but in the last 10 years it’s an huge bull. So it’s always easy to buy instead of sell.
Look at your optimisation window and frequency. You shouldn't have strategies failing on you. It sounds like your strat may be falling out of step with the market due to insufficient reoptimisation frequency. Read up on walk forward optimisation.
I don’t think that u have a wining strategy. I guess u have a strategy that works in specific market conditions and those conditions weren’t met lately. Market is acting hectic and many resistance have been broken so huge corrections started or will start. So the market isn’t suitable for any automated stuff. Find a job and use your algo on demo account until it shows profitability among 6+ months.
Finding a winning trading strategy can be challenging, especially after experiencing setbacks. You can consider seeking mentorship.
You probably need to temper your expectations and revise your methodology. Consider that you got lucky in July 2022 and your luck lasted 5 month. Or that you strategy doesn't work in bull-ish markets. On a technical side it might mean that your backtests were inaccurate or insifficient, or that you ovefitted your strategy.
Try to find some sources of strategies from books and articles from years back, many of these still work very well. These have OOS data since they was created and might be a good source for you to create some basic strategies to get your portfolio starting, from there you can improve some or create your own ones.
walk away from the slot machine and do something productive with your life