Due to a snafu with the way the date and time were programmed into OpenReview, and having NOTHING AT ALL to do with us getting slightly fewer papers than we wanted (well, actually ....) we have extended the deadline for the Embodied AI Workshop's Call for Papers to Friday, May 23rd, AOE (Anywhere on Earth):
Please submit your 2-page extended abstracts on embodied AI, especially related to this year's themes of Embodied AI Solutions, Advances in Simulation, Generative Methods for Embodied AI, and Foundation Models for Embodied AI!
Hey folks, I have been neck deep in preparations for a couple of workshops - the Advances in Social Robot Navigation one I already mentioned, coming up next week, but also the Embodied AI Workshop #6!
The Embodied AI workshop brings together researchers from computer vision, language, graphics, and robotics to share and discuss the latest advances in embodied intelligent agents. EAI 2025’s overaching theme is Real-World Applications: creating embodied AI solutions that are deployed in real-world environments, ideally in the service of real-world tasks. Embodied AI agents are maturing, and the community should promote work that transfers this research out of simulation and laboratory environments into real-world settings.
Today is Embodied AI #5, running Tuesday, June 18 from 8:50am to 5:30pm Pacific in conjunction with CVPR 2024's workshop track on Egocentric & Embodied AI.
Here's how you can attend if you're part of the CVPR conference:
The physical workshop will be held in meeting room Summit 428.
The physical poster session will be held in room Arch 4E posters 50-81.
The workshop will also be on Zoom for CVPR virtual attendees.
Remote and in-person attendees are welcome to ask questions via Slack:
You can see our whole schedule at https://embodied-ai.org/, but, in brief, we'll have six invited speakers, two panel discussions, two sessions on embodied AI challenges, and a poster session!
Going to crash early now so I can tackle the day tomorrow!
-the Centaur
Pictured: More from the archives, as I ain't crackin' the hood open on this website until EAI#5 is over.
The Embodied AI Workshop is coming up this Tuesday, starting at 8:50am, and I am busy procrastinating on my presentation(s) by trying to finish all the OTHER things which need to be done prior to the workshop.
One of the questions my talk raises is what ISN'T embodied AI. And the simplest way I can describe it is that if you don't have to interact with an environment, it isn't embodied.
But it's a static problem. Recognizing things in the image doesn't change things in the image. But in the real world, you cannot observe things without affecting them.
This is a fundamental principle that goes all the way down to quantum mechanics. Functionally, we can ignore it for certain problems, but we can never make it go away.
So, classical non-interactive learning is an abstraction. If you have a function which goes from image to cat, and the cat can't whap you back for getting up in its bidnes, it isn't embodied.
-the Centaur
Pictured: Gabby, God rest his fuzzy little soul, and Loki, his grumpier cousin.
So, as I've said, Embodied AI is just around the corner. But what is this workshop about? Embodied AI, of course! It says so on the tin.
But the key thing that makes "embodied AI" different from computer vision is that you must interact with an environment; the key thing that makes "embodied AI" different from robotics is that technically it doesn't need to be a real physical environment, as long as the environment is dynamic and there are consequences for actions.
SO, we will have speakers talking about embodied navigation, manipulation, and vision; generative AI to create environments for embodied agents; augmented reality; humanoid robots; and more.
Okay, now I really am going to crash because I have to fly tomorrow.
Onward!
-the Centaur
Pictured: An e-stop (emergency stop) button from a robot. Looks a little jury-rigged there, Chester.
Ok, the image is from ICRA, but I am still traveling, and have not fixed the problem on the website backend. BUT, Embodied AI is this coming Tuesday, so please drop in if you are at CVPR!
More later, I had several long days at the customer site and I am going to go crash now.
Hey folks! One more sticky post reminding y'all that coming up next week is the Fifth Annual Embodied AI Workshop, Tuesday, June 18th, at CVPR, the Computer Vision and Pattern Recognition Conference. I am the lead organizer, though you'll probably only hear me yapping for any extended length of time if you show up in the first ten minutes - I give the intro talk at 8:50am.
Hey folks, another "sticky post" for my next three events: The Nebula Conference, the Embodied AI Workshop, and the Workshop on Unsolved Problems in Social Robot Navigation!
Hey folks! I am appearing at several conventions in the next few weeks, so I'm creating a "sticky post" to let y'all know about my schedule - in case the problems I'm having with my blog software get worse, at least this will be up here to let y'all know I'll be talking about AI, robots and writing in the next few weeks!
First up is Con Carolinas, the Carolinas' longest running science fiction convention, where I will be on four panels and an author signing, talking about book openings, artificial intelligence, neurodivergence, and what's possible and what's not in science and science fiction!
Please send us what you've got! Just between you and me and the fencepost, if we get about 7+/-2 more submissions, we'll have enough to call it done for the year and won't need to extend the CFP, so we can get on with reviewing the papers and preparing for the workshop. So please submit!
-the Centaur
Pictured: the very nice logo for the Embodied AI Workshop, a joint effort of me, my co-organizer Claudia, and I think one of Midjourney or DALL-E. Yes, there's generative AI in there, but it took a good bit of prompting to get the core art, and lot of work in Photoshop after that to make it usable.
Our call for papers is still open at https://embodied-ai.org/#call-for-papers through May 4th! We're particularly interested in two-page abstracts on the theme of the workshop:
Thank goodness! At last, I'm happy to announce the Fifth Annual Embodied AI Workshop, held this year in Seattle as part of CVPR 2024! This workshop brings together vision researchers and roboticists to explore how having a body affects the problems you need to solve with your mind.
This year's workshop theme is "Open-World Embodied AI" - embodied AI when you cannot fully specify the tasks or their targets at the start of your problem. We have three subthemes:
Embodied Mobile Manipulation: Going beyond our traditional manipulation and navigation challenges, this topic focuses on moving objects through space at the same time as moving yourself.
Generative AI for Embodied AI: Building datasets for embodied AI is challenging, but we've made a lot of progress using "synthetic" data to expand these datasets.
Language Model Planning: Lastly but not leastly, a topic near and dear to my heart: using large language models as a core technology for planning with robotic systems.
The workshop will have six speakers and presentations from six challenges, and perhaps a sponsor or two. Please come join us at CVPR, though we also plan to support hybrid attendance.
Presumably, the workshop location will look something like the above, so we hope to see you there!
-the Centaur
Pictured: the banner for EAI#5, partially done with generative AI guided by my colleague Claudia Perez D'Arpino and Photoshoppery done by me. Also, last year's workshop entrance.
After almost a year's worth of work, at last, the Fourth Annual Embodied Artificial Intelligence Workshop is OVER! I will go collapse now. Actually, it was over last night, and I actually did collapse, briefly, on the stairs leading up to my bedroom after the workshop was finally done. But don't worry, I was all right. I was just so relieved that it was good to finally, briefly, collapse. A full report on this tomorrow. Off to bed.
-the Centaur
Pictured: A rainbow that appeared in the sky just as the workshop was ending. Thanks, God!
Hey folks, I am proud to announce the 4th annual Embodied AI Workshop, held once again at CVPR 2023! EAI is a multidisciplinary workshop bringing computer vision researchers, machine learning researchers and roboticists to study the problem of creating intelligent systems that interact with their worlds.
For a highlight of previous workshops, see our Retrospectives paper. This year, EAI #4 will feature dozens of researchers, over twenty participating institutions, and ten distinct embodied AI challenges. Our three main themes for this year's workshop are:
Foundation Models: large, pretrained models that can solve many tasks few-shot or zero-shot
Generalist Agents: agents capable of solving a wide variety of problems
Sim to Real Transfer: learning in simulation but deploying in reality.
We will have presentations from all the challenges discussing their tasks, progress in the community, and winning approaches. We will also have six speakers on a variety of topics, and at the end of the workshop I'll be moderating a panel discussion among them.
I hope you can join us, in the real or virtually, at EAI #4 at CVPR 2023 in Vancouver!
This is a followup to my Making Computers Useful series, started all the way back in 2014. (Funnily enough, the 2013-era iMac featured in that series is now pretty damn useless as it has fallen out of update range, and locks up if you run Dropbox and Google Drive at the same time).
But, the overall goal here is to document some of the stuff that I need to do to make computers work for me. Typically, there’s a lot of friction in software, and it takes a good bit of work to make that all function on a new machine. Sometimes that becomes a deep dive.
This is one of those stories.
So today, while updating the Embodied AI Workshop’s website prior to the launch of the 2023 version, I wanted to run the tree command. Tree is great because it can help you understand the structure of a directory tree, like so:
I felt I needed this because the Embodied AI website is built on yarn and gatsby, and it turned a relatively simple site into 1.6 gigabytes of generated junk (which I noticed when one of my older computers started literally wheezing as it tried to mirror all those unexpected files):
As it turns out, you can get tree via Homebrew. Homebrew is a “package manager,” kind of like an “app store for the command line,” and Homebrew helps you get standard Linux tools, like tree, onto your Mac so you can take advantage of all the hidden Unix goodness in your Macintosh.
However … I’m a bit leery of Homebrew because this is how it installs itself:
I mean, WHAT? curl a file and run it with bash? Seriously. Now, look, I’m not saying Homebrew isn’t safe - every indication is that it is - but that this METHOD of installation is a recipe for disaster.
Why? Well, in case you’re not in the know, what this installation instruction is suggesting is to DOWNLOAD RANDOM CODE FROM SOMEWHERE ON THE INTERNET and RUN IT ON YOUR COMPUTER WITHOUT CHECKING IT AT ALL!
Nothing can go wrong with this plan.
Now, I’m no expert, but I’m familiar enough with this stuff to know what I’m doing. SO, first I checked with a few quick searches to see [is homebrew for mac safe] and it appeared to be.
SO I downloaded the software with JUST the CURL part, like so:
Folks, seriously, never do this on a site you do not trust.
After I had the code, I then inspected this homebrew-install.sh file to find out whether it was safe. I didn't see any obvious malware, but when I ran it, it wanted me to TYPE MY PASSWORD.
Seriously?
Please, I’m asking you, do not hot-pipe random software straight off the internet and run it straight from the command line and give it your password when it asks. If someone intercepts the website, and gets your password, they can do anything.
(SERIOUSLY. Once I was working with a legitimate Google representative about a Google ads program and when I went to log in to Google ads to check something, a hacker injected a fake Google ads site between me and Google, and damn near got my password. Only two-factor authentication saved me, as it broke some key link in the chain.)
BUT … it is the PATTERN I’m talking about here, not the specifics. Everything I’ve seen about Homebrew says that it is safe. I’ve even used it before, on other machines. SO, after some more research, and a little more code analysis, I confirmed this password-asking was safe, and gingerly went ahead.
And it went fine.
I had to pay thirty million bitcoin to a Russian spammer, but I wasn’t using it anyway, and I’m sure at least they got to buy a cup of coffee or something with it. :-D
Seriously. It went fine. And I love Homebrew. I just go through this every time I need to “bash” run a piece of “curl”-ed software straight off the Internet and then it asks for my password.
Still, tree worked like a charm. (Screenshots of its use were above). There are more pieces of Homebrew software I need to install, but as one test, I tried to install “banner”, a program to create oversized pieces of text, which I use in scripts to alert me that a big task is done.
But, it seems like Mac already has a version of banner, which works differently on Mac than Linux, printing VERY large ASCII banners that are vertical rather than horizontal. That’s useful, but not for my case, so I dug around for an equivalent tool. brew install figlet is the way to go:
All great!
It didn’t help me with my work on the Embodied AI website, as I had already moved on to fixing other problems on that website, and was only “brewing” things in the background while I did other tasks (like remote-attend the church vestry retreat).
But removing this friction will help me in the future. The next time I need to examine the tree structure of a directory, it's one command away. I can put banners in my scripts. And I can easily add new software with 'brew' the next time it comes up.
AND, as a bonus, I discovered a site which is doing something very much like what I want to do with the Making Computers Useful series, Sourabh Bajaj’s Mac OS Setup Guide, which “... covers the basics of setting up a development environment on a new Mac.” I have an internal document called “Mac OS X New System Tasks” which documents for myself the travails I go through every time I get a new Mac, and Sourabh’s guide seems like it provides a public version of what I want to do. Which is great! Less work for me. ;-D
On to the next task!
-the Centaur
P.S. As another added bonus, I composed this in Google Docs, and pasted it straight into Gutenberg, the new Wordpress block editor. It worked like a charm ... EVEN DOWN TO PASTING IN THE IMAGES! If this is a feature of Gutenberg, I will have to consider saying my favorite three words about it ... "I was wrong."
P.P.S. Don't hold your breath on that, though, I'm waiting for the other shoe to drop.
After almost seventeen years at Google, I've made the difficult decision to get laid off with no warning. :-) Working with Google was an amazing experience, from search to robotics to 3D objects and back to robotics again. We did amazing things and I am proud of all my great colleagues and what we accomplished together.
However, my work in robotics is not done, and I will still be pushing for better robot navigation, large language model planning, and especially social robot navigation and embodied AI. I'm spinning up an independent consulting business and will announce more details on this as it evolves - feel free to reach out directly though!
-the Centaur
P.S. Sorry for the delay - this has been up on my Linkedin forever. But for some reason I just wasn't ready to post this here. Avoidance behavior, however, has gone on long enough. Time to move on.
Pictured: me and Ryan at Sports Page, the traditional hangout you go to on your last day at Google. It was a blast seeing all the friends, thank you for coming!
What happens when deep learning hits the real world? Find out at the Embodied AI Workshop this Sunday, June 20th! We’ll have 8 speakers, 3 live Q&A sessions with questions on Slack, and 10 embodied AI challenges. Our speakers will include:
Motivation for Embodied AI Research
Hyowon Gweon, Stanford
Embodied Navigation
Peter Anderson, Google
Aleksandra Faust, Google
Robotics
Anca Dragan, UC Berkeley
Chelsea Finn, Stanford / Google
Akshara Rai, Facebook AI Research
Sim-2-Real Transfer
Sanja Fidler, University of Toronto, NVIDIA Konstantinos Bousmalis, Google
