Script progress

Published:

Updated:

The script works!

Feels like an accomplishment already, even though the limited test was for the first 100, everything worked as expected!

Every day seems like new lessons are learned. I removed the innermost function because I learned that in Python, inline code has way less overhead under the hood than function calls.

Last thing to do is run the test script with some timing code to accurately calculate the loop times.

There are 8 csv files containing draw histories and there are 8 output csv files that will hold the top 3 lists for each column for each game.

I decided to scrap the email notifications as well as I do not currently have the time to learn another library. 

Streamlined and simple. Once I have the times, I can continue to seek out more optimizations.

I do not have time info yet, but that is relatively easy to set up.

From the first draft prediction of 9,000 years, I brought the pre optimized code down to 9 years. 1000x sounds impressive, but I need to do better.

I also need to run the test script on the Pi itself to get a true prediction. So I have made huge progress, but not quite ready to launch the program just yet.

There are several factors that have contributed to the optimization so far...

1. Removing the inner function in exchange for inline code. The potential savings are the overhead associated with 280,000,000,000 function calls.

2. Reducing the top 10 list to a top 3. This is a 70% reduction in the size of the heap. Less elements mean less comparisons.

3. Reducing the program screen prints to just 28 lines. No need to overwrite as this is super clean... it just prints out the game name and column name before it starts looping. It is a tradeoff for run time, I will only know which column oof 10,000,000,000 it is working on, not where it is progress wise.

That is where I am at so far... progress has definitely been made. The Raspberry Pi has been set up and is waiting for the files to be sent so it can get going. This last stretch will probably be the most difficult as I attempt to gain a realistic run time. The program works... but can it work faster?

Then there is the ever present reality that none of this makes a difference with playing and winning... but at least I will be able to say I have tried everything!

Happy Coding!

Entry #281

Comments

Avatar hypersoniq -
#1
There is yet another process to learn, using a tool called Cython, time can be saved by applying strict C language types to the data (all integers) and compiling the code to run as a C program.
I read an article where a data scientist used cython to cycle through a 1 billion iteration 'for' loop that performed simple calculations at about 1000x faster than the pure Python code.

Complexity adds to the cycle time, but a turbo boost could yet make this vision a reality!

Post a Comment

Please Log In

To use this feature you must be logged into your Lottery Post account.

Not a member yet?

If you don't yet have a Lottery Post account, it's simple and free to create one! Just tap the Register button and after a quick process you'll be part of our lottery community.

Register