hypersoniq's Blog

The Markov Decision Process (MDP), this will be the framework for Phase 2, it will define the features, the goals and the constraints, the rewards and penalties.

The actual navigation of the framework will require an AI agent driven by the reinforcement learning (RL) framework described by the MDP.

So this is actually a 2 part mission. It may include giving the agent the ability to run the phase 1 script with changing variables. This is one of those things where the agent can output what settings were used so it is not a mystery.

Confusing to be sure, but the analogy is that the RL agent is the maze runner while the MDP describes the maze.

The week was like most weeks, no matches, even boxed... however next week requires back to the out of pocket funding.

The day draw pick, 063 had a sum of 9, which was in the wide range of 8 to 18. The night pick, however is 010, sum of 1.

After finding the sums between 8 and 18 happened in 74.468% of the last 94 draws of neutrals, it might be added to the process to at least make sure the sum falls in that range... just for a week. It does have to hit the other criteria as well... closest percent to the median, mid level of draws since last appearance, neutral classification...

The 063 did not show up, closest was to a box hit on 8/28 with 016 and on 9/2 with 366.

The evening has a draw left, only 1 box close call on 8/31 with 004. Shame because a 0.50 box hit with the pair of 00 would have paid $80...

Knowing the cost for the rest of the year still puts a single $250 straight hit with a $40 box hit in that place where it returns anything I spend plus covers the entire remainder of 2025. Hoping that can happen sooner rather than later. (Or not at all)

Who would have thought that 1 in 1,000 was so hard to figure out...

Now don't get me wrong, using sums violates 2 personally held beliefs...

1. Each position in a combo is as random and independent as previous draws.

2. The numbers have no numeric properties, they could have easily used A to J as 0 to 9.

But thinking about sums... a range from 0 (000) to 27 (999) that encompasses all 1,000 combinations.

A bell (Gaussian) curve over the distribution history, with sum 13 and sum 14 at the top of the curve. In contrast to the discrete uniform distribution created by taking pure frequency.

A one to many mapping, meaning many combos at the highest peaks.

Just like the digits themselves have no memory, the sums are just as random.

So, what use could they be?

My neutral selection process could have completely different neutrals in 150 draws, or even as soon as the next week. As I recently ran a full back test on the entire history, it might be worth a look to see what the observed sum range is. A tool to be used to help with the per column synchronization problem I have in all such systems. Roughly 10% of the data presents itself as all neutral in a back test, so that would be 1,700 sum samples to look at. Is there a range in which most would sit? Would any ranges NOT be present?

The aggregation of odd and even or high and low would still be per column and not of much use, but the sum could include a third group of statistic, we have per sample and per digit, this would be per draw... used on the observed history it could provide a loose guideline to apply when selecting from each category, which ones fall into a common sum range?

Of course any such constraint increases the chance of throwing away the winning combos, but all systems seem to do that even if it was not intentional.

Might be worth the look...

Turns out, in the last 94 neutral draws, 70 were within the sum range of sum 8 to sum 18, with the highest concentration between sum 11 and sum 16. That is interesting but not directly applicable.

What about going all in on coincidence? What about a path finding algorithm that takes actions to pick the next draw (per column, of course) and gets reward points for being correct and loses points for being wrong... however, it is given the freedom to use any method to get there. It could transform (+1/-1 etc), it could mirror, it could perform arithmetic and / or algebraic calculations, etc...

Training the model on past data to arrive at a best guess BUT also output what it used to get there... 

I will learn the majority of what I need to know by the time I finish the Markov Decision Process via reinforcement learning script for the current system... 

So if a system is given a toolbox of techniques to solve the problem and let loose to do it's thing, what kind of pick would that generate?

I still have the rest of 2025 to focus on the current system, but I need to start thinking about next steps far ahead of retiring the current system...

Nothing yet.

As I proceed with the PA Lottery Results RSS feed scraper, it looks like I have to manually update the big game files to within a few weeks to make it work... that is a pain, but... needs to be done. I am also exploring using their website, as you can get a whole year at once. This would be more powerful help for catching up on the games that were not updated in a long time. Why does PA have to stay in the stone age by NOT having an API? It is MUCH easier to extract data using JSON.

I am committed to this current system through December 31... already know it will cost $252 ($14 × 18 weeks). With 3 draws left on the free play, there is still a chance to avoid any of that being out of pocket, but in the most likely outcome, it would still be one straight hit in those 18 weeks that could erase the expense, or 7 $40 box hits...

Markov Decision Process using a "reward" system is looking to be quite an undertaking... I do not yet have even a flowchart together for the process... BUT, I can output the data to multiple files (one for the model data and one with the individual digits and their features), so that is a start for the MDP script, know your input. The next step is to know your process and finally to know your output. I know the first and last...

Moving slowly forward, but still moving forward!

Gonna have to get a ticket, gonna use the Python Mersenne Twister PRNG. The free MM QP was worthless.

When dealing with these games on a per column basis, there are probably 5 of the last 7 weeks where each pick DID show up in it's respective position within the next 7 draws, however Synchronization remains a problem, as they do not end up on the same draw.

Though I could see this to a some extent in every singe system I have ever worked with, it is the eternal plague of the pick N games. In order to win, they need to line up on the same draw. But since each draw position is as independent as the previous draws, I do not have a workaround... it is why each pick will always ever be nothing more than a "best guess".

Also, follow up hits have proved to be challenging as well. 90% of every big idea I have tried to implement has hit once, then that was it. In order to implement the rest of the plan (pick 5 "Quinto" and Cash 4 Life only played on won money from the pick 3) there needs to be more than just 1 hit. I am on my last week (through Wednesday of next week) of "free" play.

Now, moving to using the inter quartile range to classify as hot, neutral or cold has lowered the frequency of all neutral observations from 85% of all windows to 70%, but it is more stable because the range is not as prone to influence by the outliers as is the standard deviation.

So, even though this system looks quite different from previous ones, it still suffers from the same synchronization issues, which contributes to the low hit frequency issue. The Cash 4 Life only needs to work once... the pick 3 was always meant to be the engine that propels the system forward... I have yet to solve that problem.

The addition of the chi square statistic and the P value was a success. What is of interest is the P value, because it goes from 0 to 1, the higher it is, the more representative your observed distribution is to the expected distribution, which is a discrete uniform distribution. A P value of 0.05 or less means the distribution does not match... something is not right with the results... in 2,400+ runs in the sliding back test, the lowest observed was 0.13 and the highest 0.97, so that tells me the PA lottery draws DO fit the expected uniform distribution. The variance will be left to the next script to figure out.

All 4 pick N scripts are updated to include the latest updates, which are

1. Calculating HOT and COLD by the 1st and 3rd quartile rather than expectancy +/- 1 standard deviation.

2. Adding the number of rows since the last appearance of each digit in the distribution

3. Adding a function that adds the chi-square goodness of fit test and displays both the chi square statistic and the p value for the 150 draw sample.

The 4 versions of the script are

1. The production version, which runs the last 4 7 draw windows

2. The test version that steps through each of the last 7 draws

3. The development version, which is the sandbox for developing new features

And finally

4. The sliding window full history back test. This one outputs to a csv file and allows counting of some over all stats such as how many All Neutral draws happened and in what percent of all 7 draw windows at least 1 all neutral draw happened.

There is no window step through version for the Cash 4 Life, just production, back test and development. So in that light I am better than half way done.

And I have a legit use for the scipy library!

Good coding day!

One of the main reasons I have always tried to develop a straight shot system is of course the cost. One ticket is cheaper than a group of tickets. However, one ticket is a pain to calculate every day. Hence my move to a 7 day "forecast" window. Same cost as a daily system, but 1/7 the work involved.

Two pitfalls I avoid...

1. Playing the night pick for the day game or playing the day pick for the night. That would double the cost, and the rarity is not worth the expense.

2. Pennsylvania's "Wild Ball". Though I would have caught 2 box hits in the last 2 weeks, I cannot justify voluntarily doubling the cost for the weak payouts.

Also why I avoid working with systems that result in a matrix or working with pairs and mirrors.

At the pick 3 level, I find it more entertaining to be correct than to cash in winners that barely cover the expense of play.

Of course, on their money, expense is recalculated, such as the Cash 4 Life cash ball brute force gambit... I would never spend that kind of money on the lottery, but I WOULD spend their money...

This is my last week of house money for the pick 3, it has been a fun run for nearly 2 months.

Strategy of using the new "draws since last appearance" statistic for each digit will be altered this week. Last week I used the SMALLEST number as a tie breaker, but when reviewing the results, it appears I should have divided the longest out by 2 and used the closest to that median as a tie breaker. Q2 variance from expectancy still seems a good starting point, however I look forward to creating that Markov Decision Process script to help figure out the best interpretation of these statistics. This is probably going to take a long time. It is basically an attempt to answer the questions "Given the following statistics, which neutral number has the best chance of being drawn in the next 7 draws?" AND "which of these statistics are important, and how important is each?"

There are techniques for peeking inside of the AI "black box" to see exactly HOW it is "learning" and what it "learned"... I will need to work on those as well.

Given the biggest time soak is updating the draw histories, I am still working on scraping the PA RSS feed, since they are too cheap to have a proper RESTful API... this is one of those projects that starts with the best intentions but always ends up on the back burner when just about anything else comes up... I know what I WANT the scraper to do, read the RSS feed and store the draw info for selected games (right now Pick 3 day/Evening, pick 5 day/evening and Cash 4 Life) then read the last row of the history file and insert draws past the last recorded date... I know how to store and write the data, I just have to actually sit down and DO it...

When it comes to crastination, I'm a Pro!

So at least one of the numbers was a straight hit somewhere. The other number was a box hit in IN earlier in the week.

Last draw for the mid and eve today, then using the last of the house money to play next week tomorrow.

Interesting run so far.

The Python QP generator for the power ball, one ticket per draw for the last 5 draws with all the add ons... cost $20 (including tonight), won so far $35, net profit $15.

Progress has been made on the chi-square goodness of fit addition to the column statistics. Simple to send the function the last 150 draws using Pandas by

df.tail[col](150)

Where df is the data frame and col is the current column. Big coding day tomorrow to integrate these changes into all of the scripts.

Then, I get to start the design for the Markov Decision Process. That will hopefully see some coding progress after the flowchart is done.

So, the script produces 2 separate categories of stats.

1. Statistics of the draw sample (150 draws)

These include the variance, standard deviation, the quartiles Q1 (to set the lower classification threshold), Q2 (the median) and Q3 (to set the higher classification threshold), and after the next few days, the chi square statistic and it's P value

2. Statistics of the digits (0 through 9)

These are made up of raw frequency, percentage of appearances within the 150 draws, the classifier (Hot, Neutral or Cold), and the number of draws since it last appeared.

Now I have to figure out how to properly interpret what is shown, there are variants that have to be updated (like the last 4 windows or the back test script, and the Cash 4 Life variants as well)

The back test MUST be run to see the difference between all neutral draws per window using quartiles as a classifier vs the already run version that used standard deviation from expectancy as the classifier.

My V4 on the first script (v2 to switch to quartiles, and v3 to add the number of draws since last appearance) will add the chi-square goodness of fit stats. Once that is done, it is a simple paste of that function into the other pick N variants, and because I saved all versions, I only have to modify one of the Cash 4 Life scripts and paste the functions for the rest, that being the last 4 and the back test.

I will have to spend a few weeks figuring out if any stat has bearing on which digit to choose and why... getting down to the wire, there is $14 left on the house money voucher, so Thursday is most likely the last week before I begin picking up the tab again. A single straight win between now and next Wednesday means the state picks up the tab for the rest of the year... no pressure...

The PA lottery does not seem to like whatever numbers I play as of late...

The odds are high, the budget is low, still entertaining to make these attempts once a week rather than every draw, so I believe I will stick with this one longer than the past attempts (I would have already been done with the older systems.).

To Quote Sonny & Cher, The beat goes on...

Very simply put, it is impossible to predict random number sequences.

In 20+ years of trying, I cannot reject the null hypothesis.

We all have our own reasons for continuing to try.

The kiosks in PA read the bet slips, and they usually have a supply on hand... usually...

I find it best to hang onto a small inventory.

The ones I have are based on the current strategy... 10 slips for the pick 3 (because they would usually be the ones missing at the retailer), 2 for the pick 5, 2 for the Cash 4 Life and 6 for the Mega.

Because of the layout, it takes 2 pick 3 slips to play mid and evening because the day/night/both option is for the whole slip, not individual games. The same is true for the pick 5.

I keep 2 Cash 4 Life because my window is 26 draws and they only go up to 10 advanced draws. So one set to 10 got played twice and one set to 6 got played once. The first attempt was not successful, but it was on the house.

The Mega Millions tickets will never get used at this rate, but if they DO get used, it will also be "on the house", though that slip allows for the full 26 advanced draws. Should that brute force attempt ever come to pass, it will be some undertaking. 

But back to the slips... I like them because it removes human entry error, also blazingly fast.

Since the only sure play I will make is the pick 3 once a week, I keep more of these on hand, and the rest is bare minimum just in case the opportunity presents itself.

Since this system began, I have used the Cash 4 Life set once, the pick 5 twice (also no luck) and have not touched the Mega Millions slips.

The plan for the mega slips, there is a post it on the back of each, with letters A through F. They are for identifying the mega ball range.

A = 1, 2, 3 and 4

B = 5, 6, 7 and 8

C = 9, 10, 11 and 12

D = 13, 14, 15 and 16

E = 17, 18, 19 and 20

F = 21, 22, 23 and 24

Why only use 4 per slip? Because at 26 draws, each game is $130 @ $5 per ticket, this keeps the total to $520 per slip, so 2 $250 straight tickets and one box ticket cashed in for each stays under the $600 claim form radar. Each one being played at a different kiosk (8 retailers within a mile, so one trip). And the ONLY way this gets played is on house money. Keep in mind at 50 cents straight and boxed on the pick 3, funding this will require 12 wins, so probably NEVER going to happen, but if it does, the plan is in place!

Since Pennsylvania does not seem to like to pick my numbers, maybe the coincidence engine will kick in for another state.

Would only let me post starting tomorrow so they are up for 5 draws.

Mid Day 903

Evening 082

As I wonder how to link phase 1 to phase 2, I am thinking phase 1 output to a csv file which the phase 2 script will read. I like this work flow concept because phase 1 is starting to get too complex to build on and still understand the operations. In this way, phase 2 can be a clean sheet design. I may go through some tutorials on building software for the Markov Decision Process and also on the implementation of reinforcement learning so I can synthesize a solution that "learns".

So now running the phase 1 displays statistics for each column...

1. The first, second (median) and third quartiles

2. The standard deviation

3. The variance

This section will eventually also contain the P value for the chi-square goodness of fit test.

Then, the output that was calculated for each digit in each row is displayed side by side.

The digit, the frequency, the classification (H, N or C), the percentage of the digit's frequency to the whole model (150 draws), and now the number of draws since it's last appearance. This is repeated for each column. So 0, for example, shows the data for 0 in the first column, then the second column and the third column. This stretches out 2 more columns when looking at the pick 5.

The new metric of last appearance was easy to test, by comparing the number given to it's position in the history file, for each column.

Now, I need to figure out how to interpret the data. This first week the following scenario was used...

1. Look at the percentages closest to the median.

2. If there is a tie, use the lowest last appearance as the tie breaker.

So going forward, this will be evaluated each week, to see if things need to be changed. Maybe furthest out is better? Maybe instead of the median as a guide, look at numbers about to transition classifications? There will be plenty of time to experiment until I have something resembling a phase 2 implementation... but the all neutral QP generator will be retired for the time being.