
Long day. But I had yet another victory with "push it just a little bit farther" combined with "nailing down the carpet", applying them together to successfully complete a data loader for my latest machine learning project. It was quite the mess at first, with loose wires and dangling bits all over the place, and while the high level concept of what I wanted to do was clear, some of the next steps were elusive.
But "nailing down the carpet" means methodically going through a project and eliminating everything that can trip you up - formatting files, turning on the linter, resolving lint issues, refactoring code, and, sometimes, just moving code to its proper place. And when I was done with that, my data loader class was practically empty, just waiting for a suggestion from ChatGPT to flesh it out.
I had to adapt that code to my use case, of course, but I successfully loaded my data (into a Colab which was now a third of its former size thanks to my aggressive moves of code into reusable libraries) and managed even to cut the proposed loader to half its size, again due to the reusable libraries I had just built. The code worked in Colab. And I wanted to check it all in - but the unit tests suggested by ChatGPT no longer passed after all my code changes. It was late and I was tired, so I decided, yeah, time to hang it up.
But I was so close. And so, I decided to "work a little bit harder," and fix the unit test. Once I dug into it, I realized the problem was the synthetic data that the generative AI had proposed in the unit test, so I replaced that with real data, using the librarized code I'd just refactored. And then I realized the data was too big, so I used ChatGPT to write, on the fly, some code to squeeze the data down to size as test data.
That extra work took less than an hour - maybe less than thirty minutes. But it meant I was able to package up a report to my team and toss it over the virtual cube wall, confident that I had a clear picture of the data they were sending me and a clear set of tools to deal with it. And my next step, after a couple of minor refactors, is to finish the data loader so it can look at sequences of frames - something that we strongly suspect is needed to solve this machine learning problem.
So, once that's done tomorrow ... it's on to learning.
Don't jinx it, Francis.
-the Centaur
Pictured: Loki, being very comfortable in the Captain's chair. And so my point, and I guess I had one, is that by pushing it a little bit farther, almost past my comfort zone, I in turn made things so much more stable that I am actually more relaxed and calm than I was when I was planning to turn in early. So I find the tools that I'm developing - "nail down the carpet", "sharpen your saw", "work a little bit harder", "clear the decks", "find the price and pay it", and "be gentle with yourself" - continue to reap greater and greater rewards.