Press "Enter" to skip to content

The Embodied AI Workshop at CVPR 2021

centaur 0
embodied AI workshop

Hail, fellow adventurers: to prove I do something more than just draw and write, I'd like to send out a reminder of the Second Embodied AI Workshop at the CVPR 2021 computer vision conference. In the last ten years, artificial intelligence has made great advances in recognizing objects, understanding the basics of speech and language, and recommending things to people. But interacting with the real world presents harder problems: noisy sensors, unreliable actuators, incomplete models of our robots, building good simulators, learning over sequences of decisions, transferring what we've learned in simulation to real robots, or learning on the robots themselves.

interactive vs social navigation

The Embodied AI Workshop brings together many researchers and organizations interested in these problems, and also hosts nine challenges which test point, object, interactive and social navigation, as well as object manipulation, vision, language, auditory perception, mapping, and more. These challenges enable researchers to test their approaches on standardized benchmarks, so the community can more easily compare what we're doing. I'm most involved as an advisor to the Stanford / Google iGibson Interactive / Social Navigation Challenge, which forces robots to maneuver around people and clutter to solve navigation problems. You can read more about the iGibson Challenge at their website or on the Google AI Blog.

the iGibson social navigation environment

Most importantly, the Embodied AI Workshop has a call for papers, with a deadline of TODAY.

Call for Papers

We invite high-quality 2-page extended abstracts in relevant areas, such as:

  •  Simulation Environments
  •  Visual Navigation
  •  Rearrangement
  •  Embodied Question Answering
  •  Simulation-to-Real Transfer
  •  Embodied Vision & Language

Accepted papers will be presented as posters. These papers will be made publicly available in a non-archival format, allowing future submission to archival journals or conferences.

The submission deadline is May 14th (Anywhere on Earth). Papers should be no longer than 2 pages (excluding references) and styled in the CVPR format. Paper submissions are now open.

I assume anyone submitting to this already has their paper well underway, but this is your reminder to git'r done.

-the Centaur

Day 139

centaur 0
cinnamon frost

Sketch - I mean, full sketch, like with roughs and rendered inks and stuff, not a 15-minute Sharpie exercise - of Cinnamon Frost. Other than forgetting her whiskers, I think this came out well. While I did use a reference image, it's not precisely the same character (Cinnamon has way more voluminous hair, almost but not quite an afro), so it's hard to judge how well it came out with regards to proportions et al: girl headscarf

Mirror reflecting it, it doesn't look too bad. Looking over it, there's a little weirdness with the exposed shoulder being too far out compared to the size of the head and the shape of the chin, but in my defense, I was focusing primarily on the hair and headscarf, and the shoulders were an afterthought in the render.

tsorf nomanniC

Drawing every day.

-the Centaur

Day 138

centaur 0
sarah connor sketch

Super quick Sharpie sketch (yes, I got a new shipment of Sharpies) of Sarah Connor from Terminator: Dark Fate. What happened to her jaw, man? Wow.

sarah connor headshot

No easy way to make this one line up, even with distortion to try to make the proportions better. I can chalk part of that up to quick sketching with no roughs - once that Sharpie line is down, it's down - but there's also missing that tilt to the head, and squnching the features of the face.

sarah connor comparison

But it's late and I'm tired.

Drawing every day.

-the Centaur

Day 137

centaur 0
lucy scribble

I would call this a quick sketch of Scarlett Johansson from Lucy, but I am so tired I actually started fading as I was doing my quick sketch, so I quit that and did an even quicker scribble. Real drawing tomorrow.

Drawing, even if just a scribble, every day.

-the Centaur

Day 136

centaur 0
alien isolation sketch

Quick Sharpie sketch of the cover of the Alien: Isolation game, run through Photoshop to create the lens flare - and, secondarily, the background greys needed to make the lens flare pop, and the green photo filter to recreate the overall look. I made the mistake of rotating the head and helmet as I drew it, but, hey, that was easily fixed in Photoshop too. Overall, it's rough and sloppy, but not too bad, though the original face shows a lot more fear, especially in the eyes and a little bit in the shape of the mouth:

alien isolation original

Drawing every day.

-the Centaur

Day 135

centaur 0
loomis hands

Clasped hands from the title page of Drawing the Head and Hands by Andrew Loomis. While this was from a full drawing after pencil roughs, I did simplify the rendering to use just five primary levels of value (white, black, two levels of crosshatching, plus the ink outlines of course) to make it easier on me.

The outcome: not ... terrible, but not great. I'm not going to include a scan of the original as it is inside the book, but for comparison, here's my attempt at this drawing from two and a quarter years ago:

loomis hands earlier

Admittedly, this drawing was much smaller than the new one, but the old one is still pretty sloppy. It does have a nice energy to it, and the dark outlines I use as a crutch make the old drawing pop.

Still, both of these fail to catch something about the barely visible palm of the left hand (in this picture, the left hand is on the right side of the drawing, and the palm is just barely visible at the edge of the index finger) which shows up perfectly fine in Loomis's drawing with just a few lines. This is definitely one of those times where flipping the drawing 180 makes it easier to see the true shape.

Maybe that's a sign of a really good drawing: it can look better when rotated or mirror reflected than the original. I sure have a long way to get there.

Drawing every day.

-the Centaur.

Day 134

centaur 0
day 134 centaur

Tired, could stay up later to finish a full drawing, but then, I've been having trouble getting to sleep once in bed when I do that, and I don't want to have another bout of awake-till-6am insomnia. Here's a quick sketch to tide you over - with a brush pen, since I seem to have exhausted all my Sharpies.

What I did instead this evening was art related: I hung some of my wife's paintings in the new place:

sandi paintings 4
sandi paintings 3
sandi paintings 2
sandi paintings 1

Now that's art. As for me, I'm still drawing every day.

-the Centaur

Day 3, Vaccine 2, Drawing 133

centaur 0
centaur tired Mostly vaccine recovered, but didn't sleep well. Pretty tired, crashing out early. Drawing every day. -the Centaur

Day 2, Vaccine 2

centaur 0
an ouch in the arm

Also Drawing Every Day #132, but you probably guessed that.

Go, immune system, go! Back to bed.

-the Centaur

Day 131

centaur 0
bond sketch

Quick Sharpie Sketch of James Bond from the very best Bond movie, the 50th anniversary special Skyfall . Eh, meh. To me, it's recognizable as James Bond, but not as Daniel Craig:

bond macau

I'm sure I'm missing some fine details, but one big problem is that the proportions are uneven. Handling this by roughly matching height and shrinking the width to fit, the neck, collar and tie proportions are roughly 15% too wide, whereas the ears are about 5% too wide:

bond sxs

Got to crash early, so hopefully will resume work on Drawing the Head and Hands tomorrow.

Drawing every day.

-the Centaur

Day 130

centaur 0
2021 head sketch

Sketching (pencil and ink) skulls from Andrew Loomis's Drawing the Head and Hands. This seemed familiar, and as it turns out, checking through my files, I had actually tried this exercise of rebooting my drawing skills before, in early 2019, without the forcing function of "Drawing Every Day".

2019 head sketch

In some ways, I seem to be getting better, but in others, I feel I am getting worse. I know I'm specifically optimizing for quantity over quality, and on top of that, optimizing for enforcing the habit and getting over my embarrassment over doing a good rendering - using a Sharpie if I have to - but still, some things just seem worse to me now. I know I was limited by the above to what I could carry with me (back in the days when I did this over lunch, usually out) and that I'm cheating on the above by explicitly using extra-dark lines to emphasize the outlines, which is a crutch I am trying to stop leaning on; but still, compared to my recent drawings, I feel like I was getting closer in some ways to what I want.

2019 woman and child portrait

I skipped the hands and cartooning sketches this time through the exercises, though I may loop back to them. Nevertheless, I can see good things and bad things about my more realistic earlier renders. The renders are definitely nicer than the quick Sharpie sketches, but, for example, the eyes below are too big compared to the original, and the hair is if anything too large. Clearly, I need to keep up the practice.

2019 face portrait

Nevertheless, drawing every day.

-the Centaur

Day 129

centaur 0
mcgrath sketch

Quick Sharpie sketch of Patrick McGrath as photographed by Sally Soames, taken from the book Writers. (Mostly because I didn't have time to do a full start on the next page of Drawing the Head and Hands). No roughs or anything; just a straight sketch. While there are obvious errors, the hardest part was figuring out what to render as black or white against this fairly dark black and white photograph:

mcgrath soames

A direct comparison shows there's no easy way to line up what I drew with what was actually there. The nose and mouth fit best, leaving the hair and chin almost not terrible, but the cheeks are lopsided, the eyes too high, the forehead just ... wrong ... and the left shoulder shoved down while the right shoulder is up, I think because the hair came down too far. Not sure how I missed that, but it's just way off.

Also that weird "1 line out of 3-5 goes kazoo" is in full force, though in reality it's more like 1 line out of 20-30, lulling me into a false sense of security before the chin goes all wonky just when I think I've got it all under control.

mcgrath comparison

This is why roughs are important. Next time (when I'm not already up late due to work).

Drawing every day.

-the Centaur

Day 128

centaur 0
loomis frontispiece sketch

A sketch of the frontispiece of Loomis's Drawing the Head and Hands. Not going to include a scan of the page for comparison, as you should buy Loomis's book for that. I tried and abandoned 3-4 different half-finished roughs, finding problems like: squashed heads, bad jaw lines, squashed heads, and ennui. In the end, I flipped it upside down to get the "landscape", then back over to get the details.

While this isn't terrible, a few things stood out:

  • First, I need to work on my patience. "Real" artists work over the entire surface that they're rendering, and while I started to do that here, I lost patience before I reviewed the child's eyes (though, in my defense, I was trying to do a quick sketch after those 4 false starts ate my time).
  • Second, the wandering line thing I have going was in full force here: several times my lines just jumped, and while I noticed that flipping the page around to get a better angle helped, sometimes the line I drew was what I wanted, and other times it just fricking wasn't. It's really weird, as sometimes I can create really long, precise lines, and others ... zoom across the page.
  • Third, I still have trouble with the relative size of eyes, and with the fine details of nose and mouth, but especially, the size and fine details of the eyes. Also, my eyes suck. Eyebrows can use some work too.
  • Fourth, and this is a general thing, my shapes are primarily 2D, not 3D. This makes everything too flat.

Another thing, not seen in the finished work, is that I noticed I was making things too symmetrical; real faces almost always have a slight yaw around the neck, and that tilt means the eyes are closer to one side or the other; this meant I often was squeezing / moving the eyes around incorrectly, trying to create the wrong amount of space. Another thing to look out for.

Also, and I'm happy about this, I've noticed some triangular formations around the eyes / nose (I have seen these in art books) and the lines of the mouth (not seen in art books, but you see a little of it in the shapes of the mouths in the Simpsons) as well as curved rectangles between the lower lip and curve of jaw, and to a lesser degree between eyebrow and hairline on some people. All put together, they helped me get the landscape in place better this time, making the overall thing look less borken.

But not completely. Once mirror-flipped, some of the lopsidedness is more clear, especially on the kid:

loomis reflected

Ugh. Well, back to the drawing board.

Drawing every day.

-the Centaur

Day 127

centaur 0
loomis head sketch

Quick Sharpie sketch of the cover of Drawing the Head and Hands by Andrew Loomis. Whenever I'm having trouble grokking something, I usually do best if I go back to the beginning and review my fundamentals, so I searched through my collection of drawing books and started with this one. I like Loomis's work: while it is old school, he draws well and communicates well about how to draw. In this case, I'm going to try to methodically go through the book, as I started to a while back with his Figure Drawing For All It's Worth, which is perhaps my second favorite art book after Wizard's How to Draw: Getting Started.

loomis head original

In this case, I haven't even cracked open the book, or even tried roughs: to free up some time to chill on my Saturday night, I just started with a quick Sharpie sketch to warm up based on the cover. It's tricky, as you need to make bolder choices on what is shade or not when using a Sharpie, and Loomis is using several levels of value here. Nevertheless, the result is ... not terrible, though Sharpie resolution limits the drawing, and I missed some delicateness of the face's features. But on a closer, side-by-side inspection, you can see some of the flaws more clearly:

loomis head comparison

Lining up the nose and mouth (which matched best, shown in light grey above) revealed three or four things right off. First, I had missed a tilt to the head (corrected in the side-by-side above). Second, I had made the mouth too small and high compared to the jaw (in the correction, this shows up as the jaw being too low, but the real problem was the reverse, as I started with the jaw; the eyes being too high is another part of this overall misestimation of facial features). Third, the hair is too small, showing I'm decent locally, but not great at getting the overall page distances - what I call the "landscape" - correct. This means a feature may be OK, but their relationships may be bad.

Fourth, I'm still having control problems on drawing lines. You can see this most easily with the left eyebrow. Three out of four of my lines land where I want them to, and the fourth seems to pop to a wholly different place. Perhaps this is just a need for practice, practice, practice, but given that I have a history of RSI issues, I plan to keep an eye on this.

Nevertheless, I enjoy the Sharpie sketches, because they're quick, you have to commit, and you get near-immediate feedback about whether the ideas you're trying are working, as opposed to various forms of full render, where I can get lost in the trees without realizing I've set the forest on fire.

Drawing every day.

-the Centaur

Day 126

centaur 0
gadot sketch

Not quite so quick sketch of Gal Gadot, roughed in non-repro blue. I have to say, I'm not so happy with this one: the thing I do where the face gets squashed is full in force here, this time even drawing the hair down on it. And this is with me working on the face proportions a few times to avoid that.

gadot headshot

Even flipping it upside down didn't help. The problems are clearer to see here:

gadot sxs

The hair's too big at top and too small at bottom and the whole face gets pulled down and to the left. Again I think experimenting with tracing might be a good thing to do here to help me re-learn the landscape, and now that Camp Nano is over perhaps I can do that.

Drawing every day.

-the Centaur

Viiictory, Twenty-Nine Times

centaur 0
camp nano april 2021 SO! Once again, I have written more than 50,000 words in a month - this time, on Dakota Frost #7, SPIRAL NEEDLE, which is close to being finished. (Yes, yes, YES, I know, Dakota Frost #4-#6 and Cinnamon Frost #1-#3 are not edited yet, editing is harder than writing, and pays less than teaching robots to learn. I'll get to them, I'll get to them, I promise). I can't figure out the new Camp Nano interface to make it cough up the usual winner banner, so you'll just get that screenshot instead. camp nano april 2021 b This is my twenty-ninth victorious Nano challenge and thirty-first attempt overall. That's great stick-to-it-ness, but I was behind for much of the month, not getting my feet under me until the 10th, but I managed a big pushes two weekends a go and a huge push last weekend, leading to me briefly getting ahead of the game right around the 28th, making today an easy coast (1500 words finished me off, though I wrote through to a notch over 1,667 words just for completeness). According to my records, that 8,154 word push on the 25th was the second most I've ever written in a day, topped only by my 9,074 word mad push to finish PHANTOM SILVER, Dakota Frost #5, on July 30th, 2016. camp nano april 2021 c Overall, a bit behind this month, which was pretty rough OKR (Objective / Key Result) planning at work. I love the IDEA of OKRs - say what you want to do (Objective, for example, write roughly 1/3 of a novel) and how to measure it (Key Result, for example, 50,000 words in the month of April), but this time it took us until almost the 20th. 3 weeks is way too long to spend on planning for a quarter's worth of effort. OH, almost forgot, an excerpt:
The questing metal fingers of the Plague Witch's "broom" branched and lunged at me. The Salzkammergutschwert’s black blade swept through the metal spikes, as cleanly as a Larry Niven variable sword through tissue paper. The Plague Witch recoiled, whirling the broom-thing, striking its black kettle end on my overextended sword hand. The Salt Chamber Sword sang out across the street, slamming into a fire hydrant in a hiss of water. But that movement naturally carried me forward, as I thought it would, and the moment the Plague Witch raised her head, I shoved my free hand at her, jamming onto her pointed beak a magical silencing wreath made of glowing vines and Technicolor feathers. “Oh, shaddap,” I said, drawing the wreath tight just as she tried to scream. The Plague Witch squeaked—she had a mask, not a beak, so the wreath couldn’t actually shut her mouth, but it could effectively gag her, and as she flailed her head, I kicked her. “And siddown!” And as she stumbled back, for a moment, I thought it was going to work. The Plague Witch writhed. I seized the Waystaff. Nyissa seized my arm. “I suggest retreat!” she cried. “No argument!” I yelled back—but retreat was not so easy. The silencing wreath wasn’t a free design, like my bluebirds or butterflies, but was an ad-hoc construct made from—and attached to—my vine and peacock tattoos, which tugged at me. “Some difficulty!” “Dakota!” Nyissa cried, pulling me away. “Let go!” “She’s got me,” I said, my feet slipping on the street. Oh, this had been a bad idea: as the Plague Witch struggled, the wreath self-replicated, drawing more and more silencing power from her own strength—but the design was imperfect, and was reeling me in towards her. “Nyissa!” Then things happened very, very quickly. Nyissa—my bodyguard, my bride-to-be, my love—darted forward, seized the Salt Chamber Sword in a burst of spray, and swung wildly at the tattoo vine connecting us. But the Plague Witch, flailing, swung her damaged broom at Nyissa—impacting her stomach. Nyissa didn’t even scream: she just doubled over in a splash of blood. The broom swept through her as the Plague Witch stumbled away, her body taken through a forward tumble, the lethally sharp sword falling from her hand—and severing the magic-tight tattoo connection. My vine snapped back to me, hurling me to the pavement. My wind went out. The Plague Witch tore off her disintegrating crown of vines, and screamed—
-the Centaur

Day 125

centaur 0
twigs sketch Quick sketch of FKA Twigs. Despite my best efforts redrawing the face 2-3 times in non-repro blue, her features swam towards the bottom right of her face, and her jaw isn't angular enough. Features being good relative to each other but poor with respect to the face seem to be one of my problems. This one might be a good candidate for a trace of the picture in vellum to see the difference between the lines I drew and the lines that are actually there (insofar as lines exist in pictures, which they sorta don't). twigs getty Drawing every day. -the Centaur P.S. 300+ words so far, will try to push a little bit more before crashing. Only ~2700 words to go for the month.

Day 124

centaur 0
aahnold sketch Another quick sketch of Arnold Schwarzenegger, again roughed with non-repro blue, but this time more properly rendered with Sakura Graphic 1 and Micron 03 and 08. I was wondering if the sketch would turn out better if I properly rendered it, even quickly, using finer instruments than a Sharpie. aaahnold headshot Head tilt is not quite far back enough, but ... it looks better rendered in something less blunt than a Sharpie. Drawing every day. -the Centaur P.S. And 2000+ words, only ~3000 left for the month. Go nano!

Day 123

centaur 0
arnold sketch Quick Sharpie sketch of Arnold Schwarzenegger, roughed first in non-repro blue as I really wanted to work on the eyes and landscape first, then cleaned up and very lightly shaded in Photoshop as it's late and I want to turn in. Eh, the eyes are still a notch too big, and lopsided, despite my efforts, but, it's not as completely terrible as some of the other sketches. arnold headshot Drawing every day. -the Centaur P.S. and 2400+ words too.

Day 122

centaur 0
stargate sketch Suuuper quick Sharpie sketch of a character from Stargate. Not terrible, but I think I need to focus on getting that egg shape of the human head a little bit more consistent (and one of the eyes got squished, but then, it's a suuuper quick Sharpie sketch without any roughs). stargate headshot Drawing every day. -the Centaur P.S. Wrote 3600+ words today (and 8100+ words yesterday, when all was said and done).