Save some data for later
Check a fixed rule on data you did not use to choose it.
The idea
It is easy to keep changing a rule until it looks good on the data in front of you.
Keep a later block of data aside. Use it only after the rule is fixed.
A small example
Imagine you have six ordered blocks of data.
- Use the first four to develop your rule.
- Keep the final two for a later check.
- Write down the rule before opening those final blocks.
These numbers are just an example, not a recommended split for every dataset.
Try it
You open the final two blocks, dislike the result, and change the moving-average window.
Are those blocks still an untouched check?
Check your answer
No. They have now helped you choose the rule.
Record what you changed. You need another untouched period for a fresh evaluation. Also check that the future outcome for a row does not accidentally cross into the data you intended to hold aside.