SO! I went "outside my circle" today and did something different, and was about to blog about "if you do what you always do, you'll get what you've always gotten" ... but as I started to write, I had this funny feeling that I'd written about that before, and sure enough, I'd blogged about it almost exactly a year ago.
Now, I was outside of my circle today because of Lent - it's Ash Wednesday, and I decided to drag myself out to an Ash Wednesday service at the church I got married at, Saint Peter's Episcopal (the "rapture-ready" church on Hudson Road, complete with to-go box handle on top). That put me in a different physical location than normal, but it took God sending me a firetruck parked in front of one of the restaurants I would have normally fallen back to before I tried a new place - the Lost Cajun, itself part of a chain I'd been to before, but for some reason I ordered something different than normal, and got the amazing blackened catfish dish above which was far better than the things I'd previously tried there.
And, weirdly, my previous "if you do what you always do" post was also right around the start of Lent. So I wonder if there's something about the spiritual earthquake that Lent is supposed to inspire that also had sent me climbing out of ruts and seeking new experiences a year ago - or, whether that experience left echoes of memory that prompted me to try the same thing again this year.
Recently Internet guru Seth Godin blogged about “Halfway Projects”: you can get value from eating half of a pear, but half a canoe is worth less than no canoe at all. I like that. It’s a great metaphor for project management, where quitting a project just before the finish line doesn’t deliver any of its value—but leaves you with all of the costs.
Now, I misremembered Godin’s example a bit - what he actually said was “half a pear might deliver 85% of its value”. But the principle is sound: half a battery charge might let you do 85% of your work … but half a parachute is definitely worth less than no parachute at all, because it might encourage you to take risks that you shouldn’t.
For project management, though, the idea helps explain my long-running idea “work just a little bit harder than you want to.” Often, when working on a project, we get exhausted, and decide to give up - but working just a little bit harder can take us over the finish line. Our instinct to save us effort can actually thwart the work we need to do to achieve success.
For example, recently I was working on a machine learning project that just wasn’t working. We’d spent enormous effort on getting the learning system up and running, without good learning results to show for it, and the arbitrarily imposed deadline was coming up to show something impressive, or the project would be axed.
But, if you know anything about machine learning, you know most of the effort goes into data preparation. We had to modify the system to log its data, massage it into a format that was useful for learning, and spend further coding effort to speed it up so it was useful for development (taking the data load from 36 hours to 36 seconds!).
The point is, we only got the data running in mid-February, and were trying to compress months of experimentation into just ten days. Finally, as the deadline approached, I got philosophical: we’d done all the work we needed to do to start learning, and my recommendation was that the team keep working on it, with or without me.
But … I didn’t stop there.
Before the final presentation, I spent time cleaning up the code, checking things in, and getting a few of the most promising programs ready to collect “baselines” - long runs of the system set up for comparisons. And the next morning, I reviewed those baselines to present a report to the team about which one was most promising.
Long story short, one of the simplest models that we tried was actually sort of kinda working. Once I realized we had a scaling issue in the output, a simple tweak made the system get even better. I spent another hour tweaking the graphs to put the human input and the system results onto the same graph, and the good results leapt out into sharp relief.
I could have just decided that the system was a failure - but then I wouldn’t have done that extra work, making it a self-fulfilling prophecy. I call this the “Sunken Cost Fallacy Fallacy”. For those not familiar, the “Sunken Cost Fallacy” kicks in when you keep doing something that isn’t working because of the effort you’ve spent, even though you have a better option.
But you can’t “decide” that something is a better option because you’re a Decisive Decider™. It actually has to be a better option, or what you’re doing is simply throwing away the effort that you’ve spent to date because you want to throw your weight around. No, if you suspect a cost is sunken, there’s no substitute for doing your due diligence - is the project working?
If it isn’t, sure, then quit. But often, that little bit of extra work can unlock the solution to the problem. During my presentation, the team asked natural about the simple model that turned out to be the most successful - and those questions made me realize it could be improved. Over the weekend, I applied those fixes - taking merely good to excellent.
Last week, as of Thursday night, I was pretty down on the possibility of success for our project. But I did my due diligence anyway, and by Friday morning, I had a working solution. By Friday afternoon, all the team knew it – and by Sunday evening, I was composing an email outlining our machine learning “recipe” that we can build on going forward.
Quitting just before the finish line wastes all the effort you spent on the project. Before you quit, work a little bit harder than you want to and do your due diligence to check whether it is working. If it isn’t, you can stop with no regrets; if it is, you will have not just saved the value of your project - you will have saved yourself from shooting yourself in the foot.
-The Centaur
Pictured: The project team. Three-quarters of them want to try a new direction, but the old seasoned hand isn't quite so sure.
So! After that damn climate-change-induced hurricane, we had roughly fifty trees down on our property (though it may have been much more, if you count smaller trees). But this disaster is an opportunity, as newly fallen logs still have a functional immune system for a short period of time ... making it a great time to use those logs for mushroom farming!
My wife and I have been interested in mushroom farming for a while, and our friend Brandon at I See Fungi hooked us up with what we needed to get started. One of those things was a drill bit that helps drill holes to hold mushroom spawn, as well as an applicator that helps put the spawn in the holes:
After that, you can optionally use wax to seal the holes to prevent other organisms from digging the spawn out or getting into it. The messy wax, which can be heated up on your stove, or, better, a cookplate, gives the mycelium the best chance of getting established as the dominant organism within the wood.
After getting this round of mushrooms going, my wife and I had a lovely evening at Chef 21 Sushi Burger ...
... then walked around downtown Greenville, which still had its Christmas decorations up..
SO! If I got my blog running back in January, and planned to blog every day, why haven’t I been posting?
Because I also wanted to draw every day … and wanted to build a buffer.
Why? Well, let’s break it down.
First, I want to draw comic books. Yes, yes, yes, I have a webcomic called f@nu fiku, but after I broke my arm, and got my laptop stolen, and found my hand-crafted blog software stopped working, and got swarmed trying to crank out my first four novels … well, after all that, I found my confidence in my drawing had collapsed.
I never was that great at drawing, frankly, but when I was working on f@nu fiku with the goal of cranking out a page a week, I never let my drawing limitations stop me. If I wanted to have an image in my comic, I had to figure out how to draw it, no matter what. But even though I wasn’t that great, I had a level of self-confidence that let me tackle whatever I had to.
But I never gave up on comics. Not only do I want to finish f@nu fiku, I have other comics I want to draw, from Cinnamon Frost and Serendipity the Centaur stories up to and including becoming the writer-artist for Green Lantern. Obviously that last one is aspirational, but I can’t frigging aspire to become the writer-artist of anything if I am not creating comics at all.
But it’s hard to draw every day if real life intervenes (like Dragon Con, for example). According to my records, I’ve tried the “Drawing Every Day” project 3 times in the past, and never made it through the full year once - I lasted 215 days in 2021 (through Dragon Con), just one day in 2023 (the layoffs), and 135 days in 2024 (through the Embodied AI Workshop).
So, I decided to do a buffer for Drawing Every Day 2025.
So, for the first part of this year, I leaned into drawing, trying to get ahead. I decided that I wouldn’t start blogging every day until I built up a buffer of drawing every day, and in an act of quixotic hubris, I also decided to start retro-drawing the missing drawings from 2024 so that I would finish those drawings as well.
But, I wondered, how far ahead should I try to get in my drawings? Following Bill Holbrook, I guessed a month, but once you’re out of January, you need a tool to keep track of what day of the year it is. I wanted something simpler … so I started to think in terms of a simple formula I could keep in my head.
Fortunately (thanks, passage of time!) months are ordered, thus can be numbered. Call the number of the month in the year “m”. Months have a notch over 30 days on average, but for a mental formula, you want to round to even numbers to keep the math simple. So 30m is a good quick overestimate of what day it is in the year.
But 30m is a variable, vulnerable overestimate, as it is more ahead at the start of the month almost 30 days less at the end of the month. You could add 30 days, but even 30(m+1) still has this variable property. So, call the day “d” and add that to the formula: 30(m + 1) + d. And that sounds great. 30(m + 1) is guaranteed to always be more than 30 days ahead.
And … 30(m+1)+d is a treadmill. Every day, you’re just at your buffer, and every day, you can’t fail to lose focus, or you eat into your buffer. That’s no good: the point of the buffer is to get your back when major life events (like Dragon Con) happen, not to put you constantly on edge that you’re about to lose your buffer.
So I decided to add a few more days to the formula. I know I typically draw two to four drawings in a session (sometime as few as one if I am busy or have chosen something complicated, sometimes as many as five if I am sketching). So adding two more to the formula gets us to 30(m + 1) + d + 2 … a number I can easily calculate in my head, and, what’s more, add to my drawings, even if I don’t have internet where I am.
It’s not perfect - when transitioning from a short month, you can find yourself a few days behind - but it’s a number so far ahead that I can skip a day whenever I have to, confident that I will be able to get back on track with the typical number of drawings I do per day. And if I am at my buffer, I can do a “retro 2024” drawing or sketch some idea not on my drawing plan (which is a whole nother topic for a whole nother post).
So. Anyway. My point, and I did have one.
Today, I reached 30(m + 1) + d + 2 in my drawing buffer.
And so today is the day I resume Blogging Every Day, with this post.
It’s good to be back.
-the Centaur
Pictured: Where I am, drawing, and writing, and one of the drawings. And unfortunately, it's too dim to do my normal photo of my drawing for today, so I'll have to scan that when I get home.
Long day. But I had yet another victory with "push it just a little bit farther" combined with "nailing down the carpet", applying them together to successfully complete a data loader for my latest machine learning project. It was quite the mess at first, with loose wires and dangling bits all over the place, and while the high level concept of what I wanted to do was clear, some of the next steps were elusive.
But "nailing down the carpet" means methodically going through a project and eliminating everything that can trip you up - formatting files, turning on the linter, resolving lint issues, refactoring code, and, sometimes, just moving code to its proper place. And when I was done with that, my data loader class was practically empty, just waiting for a suggestion from ChatGPT to flesh it out.
I had to adapt that code to my use case, of course, but I successfully loaded my data (into a Colab which was now a third of its former size thanks to my aggressive moves of code into reusable libraries) and managed even to cut the proposed loader to half its size, again due to the reusable libraries I had just built. The code worked in Colab. And I wanted to check it all in - but the unit tests suggested by ChatGPT no longer passed after all my code changes. It was late and I was tired, so I decided, yeah, time to hang it up.
But I was so close. And so, I decided to "work a little bit harder," and fix the unit test. Once I dug into it, I realized the problem was the synthetic data that the generative AI had proposed in the unit test, so I replaced that with real data, using the librarized code I'd just refactored. And then I realized the data was too big, so I used ChatGPT to write, on the fly, some code to squeeze the data down to size as test data.
That extra work took less than an hour - maybe less than thirty minutes. But it meant I was able to package up a report to my team and toss it over the virtual cube wall, confident that I had a clear picture of the data they were sending me and a clear set of tools to deal with it. And my next step, after a couple of minor refactors, is to finish the data loader so it can look at sequences of frames - something that we strongly suspect is needed to solve this machine learning problem.
So, once that's done tomorrow ... it's on to learning.
Don't jinx it, Francis.
-the Centaur
Pictured: Loki, being very comfortable in the Captain's chair. And so my point, and I guess I had one, is that by pushing it a little bit farther, almost past my comfort zone, I in turn made things so much more stable that I am actually more relaxed and calm than I was when I was planning to turn in early. So I find the tools that I'm developing - "nail down the carpet", "sharpen your saw", "work a little bit harder", "clear the decks", "find the price and pay it", and "be gentle with yourself" - continue to reap greater and greater rewards.
So ... what the heck happened to this website for almost six straight months?
The TL;DR (too long; didn't read summary) is that moving the Library of Dresan to a new provider was a huge endeavor, so I prioritized clearing everything else going on in my life until I could focus on the move.
I had lost faith in my old web service provider. Emails delivered to centaur at dresan dot com were randomly dropped to the point I had to stop using it, and image posting was no longer possible because the provider was only giving me 25 gigabytes out of my 35 gigabytes of allotted storage. The Library had to go elsewhere.
But that involved finding a new domain provider, setting up hosting, transferring all the files, transferring the database, getting the Library's WordPress installation running on a new site with new rules and a new version of PHP, and, as a bonus, transferring all the email addresses and lists to the new domain.
And, if you've never tried to transfer 25 gigabytes of files off a remote website, you can't just "do" that. A copy of that size off a consumer-grade website will just randomly fail at arbitrary points during the transfer. I had to write an entire program to help me track this (which I plan to clean up and release on Github).
But while all that was going on, I had to replace my laptop, volunteer at the Unsolved Problems in Social Navigation Workshop, launch The Neurodiversiverse, attend Dragon Con, attend the Milford Writing Workshop, clean up after a hurricane, start mushroom farming with the logs fallen from the hurricane (which had a clock attached to it), quixotically try to get some stuff prepared for GDC 2025, prep for EAI #6, handle submitting a +66 page paper with +30 authors, and prepare for the largest Christmas ever (where we hosted two parties with almost 20 people each, and had three separate groups of houseguests).
When Christmas was finally in the rear-view mirror, I then turned my attention to webworks - first fixing the Logical Robotics website, then fixing my wife's website, and finally fixing the Library itself. It was ... exactly the ordeal that I feared it would be. Actually the WordPress part, that part, it worked fine - I had already copied the files, and had frozen the database as of my July 26, 2024 post, and ... miraculously, the website was working to serve the pages with very little issue. But posting did not work (a permissions issue). And then logging into the website quit working (an SSL issue). And then posting images quit working (which turned out to be, indirectly, an SSL issue, due to the firewall bundled with the SSL).
And so on. And so on.
Yes, yes, yes, bla bla bla, you've heard all this from website developers before. But there's a very important insight I have to share with you. Yes, we are finite creatures with limited powers, and yes, sometimes we run into problems, and yes, sometimes, we run into problems that seem beyond our abilities to solve.
But, just as we are finite, so our problems are finite. Yes, yes, yes, it's important to understand the difference between a solvable problem (cleaning out your storage unit) and an unsolvable one (as when the legendary King Canute apocryphally tried to back the tide, which is actually a dirty lie given that he knew better and was just trying to stick it to his flatterers in his court, but, whatever). But as long as you are not actually trying to turn back the tides, your problems can be solved by focusing on them, one by one.
And so that's what I've been doing for the past several months since I came back from the Milford Writing Workshop. My 2024 was hectic - because we wanted to launch The Neurodiversiverse in time for Dragon Con 2024, and because I chose to do a lot of publicity for it at the Nebulas, Con Carolinas, and Dragon Con itself, but because I chose to not cancel many other events, like the Fifth Annual Embodied AI Workshop, or the Workshop on Unsolved Problems on Social Robot Navigation, or my attendance at the Milford Writing Workshop itself - requiring me to plan it down practically to the week.
After Milford, however, I had a few months until Christmas ... and I vowed to start "clearing the decks" of my massive todo lists. So I've spent the past three or four months methodically identifying things, working to eliminate them, and moving on with my life.
It has been refreshing and freeing. I have far to go - my todo list needs a fricking one inch binder clip, and I am not exaggering one bit - but, already, many things that have been bugging me are gone, just gone, leaving me with ... that ... much ... more ... free time and ... that ... much ... less ... mental load to carry.
So, this is a very long-winded way of saying, soon, I'm going to resume blogging every day.
But ... I wanted to clear the decks, and get off my chest why I haven't been.
-the Centaur
Pictured: Snow, in the "French Quarter," our tiny little courtyard.
SO! I am behind on blogging. But my wife and I have been traveling so much this year (near constantly for five months between the two of us) that, frankly speaking, we need to focus on us time more than I need to focus on the blog. So it's going to take a little longer to get things rolling ... because other things come first.
-the Centaur
Pictured: an anniversary picture, from years ago (since the blog image uploading is still borken).
Welp, by my calendar, I'm about two weeks behind on blogging every day posts, but better late than never, eh? The Embodied AI Workshop went off quite well - we had standing room only three deep by the end - even though I was frazzled from 7am to 10pm trying to make sure things went off as planned.
And the next day, we had CVPR, which was quite the fun adventure! But, then, that evening, I spilled water onto my laptop. It promptly rebooted, then shut down, never to turn on again. Not only did that make me feel like an idiot, it put a serious crimp in the work I was planning to do during the conference.
Including blogging! Not only was it difficult to post on my phone, it was also practically impossible to start down the path of upgrading the dresan.com backend to deal with the file storage issue - and what computing time I had needed to be spent on The Neurodiversiverse. So everything ground to a halt.
So I'm not dead. But it is taking a bit of time to get things back on track. By my count I'm about two weeks behind on blogging and a week behind on art, and it looks like it will take several weeks to get caught up, back up to speed and on a regular posting schedule.
Stay tuned.
-the Centaur
Pictured: The backdrop for Embodied AI #4's scheduling poster, produced with several layers of generative AI combined in Photoshop and extended with Photoshop's own generative fill tools into the poster size. While I'm convinced we don't want to use generative AI for regular art, for this client, which was a workshop on AI featuring generative AI, we wanted the generative AI look.
Today is Embodied AI #5, running Tuesday, June 18 from 8:50am to 5:30pm Pacific in conjunction with CVPR 2024's workshop track on Egocentric & Embodied AI.
Here's how you can attend if you're part of the CVPR conference:
The physical workshop will be held in meeting room Summit 428.
The physical poster session will be held in room Arch 4E posters 50-81.
The workshop will also be on Zoom for CVPR virtual attendees.
Remote and in-person attendees are welcome to ask questions via Slack:
You can see our whole schedule at https://embodied-ai.org/, but, in brief, we'll have six invited speakers, two panel discussions, two sessions on embodied AI challenges, and a poster session!
Going to crash early now so I can tackle the day tomorrow!
-the Centaur
Pictured: More from the archives, as I ain't crackin' the hood open on this website until EAI#5 is over.
The Embodied AI Workshop is coming up this Tuesday, starting at 8:50am, and I am busy procrastinating on my presentation(s) by trying to finish all the OTHER things which need to be done prior to the workshop.
One of the questions my talk raises is what ISN'T embodied AI. And the simplest way I can describe it is that if you don't have to interact with an environment, it isn't embodied.
But it's a static problem. Recognizing things in the image doesn't change things in the image. But in the real world, you cannot observe things without affecting them.
This is a fundamental principle that goes all the way down to quantum mechanics. Functionally, we can ignore it for certain problems, but we can never make it go away.
So, classical non-interactive learning is an abstraction. If you have a function which goes from image to cat, and the cat can't whap you back for getting up in its bidnes, it isn't embodied.
-the Centaur
Pictured: Gabby, God rest his fuzzy little soul, and Loki, his grumpier cousin.
So, as I've said, Embodied AI is just around the corner. But what is this workshop about? Embodied AI, of course! It says so on the tin.
But the key thing that makes "embodied AI" different from computer vision is that you must interact with an environment; the key thing that makes "embodied AI" different from robotics is that technically it doesn't need to be a real physical environment, as long as the environment is dynamic and there are consequences for actions.
SO, we will have speakers talking about embodied navigation, manipulation, and vision; generative AI to create environments for embodied agents; augmented reality; humanoid robots; and more.
Okay, now I really am going to crash because I have to fly tomorrow.
Onward!
-the Centaur
Pictured: An e-stop (emergency stop) button from a robot. Looks a little jury-rigged there, Chester.
Ok, the image is from ICRA, but I am still traveling, and have not fixed the problem on the website backend. BUT, Embodied AI is this coming Tuesday, so please drop in if you are at CVPR!
More later, I had several long days at the customer site and I am going to go crash now.
Okay, I was flying Tuesday, so I'm just going to pretend this was an abbreviated post, something something busy busy something something flying to Vancouver something something robot consulting.
At least I didn't try to fly on an expired passport ... this time. Strange how paranoid a mistake can make you! Like how I missed a flight - two days in a row - trying to leave London, ~30 years ago, the first time due to my mistake, the second due to a train stoppage, so I now try to go to airports ~2 hours early ... and missing my flight to Comic-Con due to traffic made me paranoid enough to leave ~3 hours early in LA's rush hour traffic so I'd have time to make it through any unexpected snafus with my international flight.
But that paranoia got me there safely and on time ... this time.
-the Centaur
Pictured: Me, at some event in 2015 ... wait, I owned this scarf in 2015???
Super far behind, because we're in "the stretch" leading up to Embodied AI Five - which also happens to be the week of a site visit at one of my consulting clients. So, this past Monday, I met with them online, took care of some Neurodiversiverse stuff, met friends for dinner, then started packing to fly.
And, while I did draw, I forgot to blog. Mucha-girl disapproves.
Still, blogging every day, even if I have to backfill.
-the Centaur
Pictured: Detail of Alphonse Mucha's poster for Princess Hyacinth, incorporating, when you look more closely, a disturbingly strong right arm on the princess there - in my mind, probably symbolizing both her father, the blacksmith, and probably echoing Mucha's pro-Slavic symbolic interest in the goddess Slavia.
So one thing here to remind myself to blog about it in more details - I attended a panel at the Nebulas on "Moving Beyond Milford" which was very useful. Milford, for those not steeped in writerly inside baseball, refers to the Milford Writer's Workshop, or, more generally, a critiquing model in which a group gets together, shares stories to read in advance, and everyone critiques each other.
The key element of Milford is that each person in the group gets their turn to critique - say, four minutes - during which nobody else can speak - not the other authors, not other group members, with the exception of a facilitator who can keep things on track. And most people seem to agree that the gag rule is critical to Milford in that it helps authors to learn to take criticism - and shuts up "that guy" so he doesn't dominate the critique.
But it can cause people to pile on, or for criticism to be repetitive, or even misplaced. So people were recommending different approaches - online, threaded critique, or structured critique where you had to start off with what resonated with you in the story before critiquing it, or encouraging facilitated discussion so everyone doesn't pile on with dittos.
I had "office hours" at the Nebulas, during which I advised other authors on problems - not because I'm some super experienced author or anything, but simply because I'm an editor and publisher, which means authors who are arguably equally or more experienced than me thought they might benefit from talking to an editor and publisher about specific problems. Which we did, with a couple of people.
And, when I did, I took the advice of the "Moving Beyond Milford" panel: I reformulated my critique into a five-point breakdown:
What did I think the story was about? Reiterating what the story is about ensured that I "got" what the author was trying to do, in an attempt to head off at the pass any misunderstandings.
What did I like about the story? Identifying what resonated with you about the story helps the author understand what's working about the story which they probably shouldn't change.
What areas of improvement did I see? This is something that can crush newbie authors - or experienced authors hit with impostor's syndrome - so it's important to formulate this in terms of suggestions.
What features or turns of phrase stuck with me? These might be small things, but I think highlighting key sentences, elements of description, or ideas are important to remind authors they can be effective.
What areas could potentially use copyediting? If there are typos, grammatical errors, or other opportunities for low-level textual improvement, highlight them here.
But even though that's what I used when I analyzed the story, that's not how I presented the above material to the author in our meetings. What I did instead was use the following script (after the meet-and-greet):
Here's what I think I read. I started off by briefly reiterating what I thought the story was about, so the author knew I had read their piece (and we could clear up any misconceptions).
What do you want help with? I then asked the author to explain what areas they needed help in. If anything about that wasn't clear, I asked them to explain in their own words the problem.
Let's brainstorm solutions to your problem. Before digging into my notes, we discussed their problem in greater depth and used story structure ideas to start looking for solutions.
Let's discuss where my notes intersect with your concerns. Then, we dug into where my notes intersected their problems, focusing on the parts where they needed help.
Once we have ideas about a solution, then share the other notes. Where the other notes were still relevant, I shared them, trying to build suggestions about how to make the story stronger.
Overall, I wanted to not dive in with how I thought the story could be better, but to improve the author's experience working with the story first, then focus on how my thoughts about the story could help them.
I think this is a better approach than tackling the story proper as an entity divorced from its author. Once a story is done, we can talk about the text as an entity independent of its author, but BEFORE the story is done, it's a work in process being worked on by a real human being, asking for help.
When critiquing, put helping the author first, and worry about your personal pet peeves some other time.
-the Centaur
Pictured: Some writing advice from me, from back in the day, while blog image uploading is down.
Okay, the Nebulas are over, and I should blog about that, and I'm a day or two ahead on drawing, which I should post when I get the website backend fixed, and I'm a day behind on blogging, so I should get caught up on all that.
But I just spent about two hours pursuing, and achieving, zero inbox across all three of my major email accounts, so I am VERY tired, and I am going to crash shortly.
Zero inbox is the discipline of clearing ALL the messages from your inbox - either by handling them, or categorizing them into folders for further action. This means what comes into the inbox in the future can be more quickly dealt with (or more easily unsubscribed from).
Now, I have a LOT of email in the folders I filed - probably hundreds of messages. But I had at least twenty thousand messages built up across all three accounts, most of which were spam, promotions, social media notifications, forum posts, or other notifications which were functionally worthless.
Now, even though there are hundreds of messages to process ... they're just in the hundreds.
And that feels way more doable.
Okay I go crash now.
-the Centaur
Pictured: A blast from the past in the Atlanta Airport (while blog images are still down).
But I also have a day job, and since I was "drawing, on average, every day" and reading Neal Asher's Shadow of the Scorpion on the flight out, I took advantage of my oops-you-checked-in-in-advance-but-we-didn't-reserve-your-actual-room drink voucher and got an Old Fashioned at the soon-to-close hotel bar.
(Apologies to the hotel bar staff: I came in 30 minutes before close, just as they were cleaning up and switching over to sidework, but me plopping myself down at the bar apparently opened the floodgates, for something like half a dozen people then showed up right after I did).
At the bar, I cracked open Visual Studio Code, ChatGPT and Stack Overflow in an attempt to find a more parsimonious dataset representation for one of my clients. I'd built a horribly data-inefficient version of a machine learning dataset for them on the principle "get the fucker running so we can see whether it works" but the fucker worked, so we need now to make it at least marginally more efficient as we now turn our attention to "let's see whether this fucker can fucking scale up."
It looks like a data representation called HDF5 is worth a first shot (not that it's the best or the only, but it has C++ and Python bindings and appears simple to integrate into both our custom data set writer and into our custom PyTorch data loader). So, I did a little digging via Bing/Google to verify the best way to install HDF5 for Python (h5py for Conda, in this case) and set down to try out ChatGPT's recommended test case.
But ... the installation locked up.
Restarted the install. No dice. Then I thought it was the janky hotel Wi-Fi. Switched to my own personal hotspot. No dice. Tried a bunch of StackOverflow recommendations to fix the problem. No dice. Fifteen, then thirty, then forty-five minutes stretched by, as I tried to get a simple darn package to load.
This is, as I've said before, the problem of "molasses" in computer programming: the gummy gook which makes it impossible to do simple tasks. Another colleague called it "the novice penalty, and it's real": people who work in a domain all the time learn the tricks to make it work, but novices don't know these tricks, and struggle to do things that "experts" think are easy because they've forgotten they are difficult.
I almost gave up. But molasses needs to be fought. As I often say, oftentimes, you need to work a little bit harder than you think you need to, and when you do, you'll find that you're greatly rewarded by a breakthrough. Molasses can gum us up, but if we push through, we may find that it becomes smooth sailing.
In this case, the solution was actually to use ChatGPT's suggestion for installing the HDF5 package: 'pip install h5py', rather than ' conda install anaconda:h5py'. The benefit of doing it the 'conda' way is that the installation is in a 'Python environment' that corrals the installed software so it doesn't break anything else; but, for whatever reason, my conda environment was having trouble with that, so pip - which installs the program globally on the computer, across 'environments' - was the way to go,
From there I was able to start making progress on my dataset loader problem, and have a clear direction for the project to take tomorrow. Had I accepted the slowdown imposed by the molasses, I would have returned to this problem tomorrow with no real clue of the next steps to take, other than remembering that I had tried a bunch of stuff, got exhausted, and decided to start fresh in the morning.
Sometimes that's the right thing to do, of course. But if we can push through the molasses through to the other side, we often will be doubly rewarded: not only will we solve the immediate problem we were facing, but also will have a solid foundation to move forward on our next task.
So don't let the molasses bog you down. Push on through, and leave it behind if you can.
-the Centaur
Pictured: One from the archives - some Mathematica analysis of a problem - while the blog images are down.
In case the travel process goes kazoo, I'm nabbing tomorrow's post in today - looks like on June 6th I will be at the meet and greet space Thursday at 2:30pm (Meet-and-Greet Table E Anthony Francis Pasadena – Fountain Foyer), no matter what the sticky post says.
I will have books, and will sign them; hope to see you there.
