My DeepLens Challenge Project is called ‘Nautilus Face Tracker’. The goal of my project is to integrate the DeepLens Camera with Alexa, using AWS Cloud as the back-bone and brains of the system, to allow Alexa to recognize faces as well as learn to recognize new faces she has never seen before. My project was more of an AWS Service Integration problem rather than a Machine Learning exercise, really; but I did learn alot about the DeepLens Platform and definitely have my interest piqued in learning more about Machine Learning and Artificial Intelligence. In fact, my girlfriend also has a new interest in the subject presumably just from listening to me work on this project. She even started taking a ChatBot course last week on Coursera as a result. These are fascinating times to work in the Information Technology field (and I shan’t be outdone)!
Here’s the architectural diagram I made to describe and explain, at a high level, how I put my project together. The source code is still in a private repository while the competition takes place. The Hackathon ends on Valentine’s Day (2/14/2018).
I put this whole thing together using some fairly simple Python and nodejs scripts and a few Lambda Services. In retrospect, the AWS services and programming APIs are very easy and powerful to use.
Here’s the video I made demonstrating my project. A video demonstration of working code was a requirement for this project.
So What’s Next?
I definitely want to learn more about Machine Learning. I found some good learning resources as starting points, and Coursera has some good Deep Learning classes as well:
It was the best of times, it was the worst of times. Day 10 (for me) of the DeepLens Challenge (I first blogged about this here). I have made some progress and am now able to match face images, retrieved from the DeepLens Camera, against a face image gallery I built using AWS S3, Lambda, DynamoDB and the Rekognition Service (I used this blog post to get things setup). Using the Rekognition Service was actually pretty straight forward and easy, especially as there is a clear blog outlining how to start using it to go from. Unfortunately, working with the DeepLens Camera is not so easy at times.
Downloading Projects from the AWS Console to the DeepLens sometimes get hung up. I found that running
sudo systemctl restart greengrassd.service
on the Camera usually kicks it into gear and allows the Project to download. But the build deploy process is time consuming and fraught with missteps.
Your Project version can only go up to 9 for some reason, so I was deleting my Project when the Version hit 9. However, I ran into a bug last night where the DeepLens Camera would get Deregistered whenever you deleted an associated Project. So that meant resetting the device to put the on-board Wifi in the right state so that the device could be Registered with Amazon again. Arrrggh! And no deleting Projects until this is over!
My DeepLens was automatically updating itself putting my Camera in a bad state as the AWS Camera software was apparently incompatible with the Linux updates I was receiving. I finally figured out how to turn off the automatic updates (done when Registering the DeepLens with AWS), and followed steps to lock-in Linux kernel 4.10.17+.
This is a cool little song from the immensely talented Martin Garrix. I first heard this song at AWS re:Invent in 2016. The depth of the bass and sharpness of the sounds blew me away, not to mention the psychedelic jelly-fish visuals.
This is Day 5 (for me) of the DeepLens Challenge, which I talked about starting in my post here. I have to submit my project by February 12th or 13th. I’m making progress toward my project goal, which right now is simply to recognize a face in an image cache from a live video feed using the stock face detection model on the DeepLens device. Face and image recognition is pretty common place today, I guess, but I’m stoked to get something similar working myself. I’d also love to integrate Alexa into the mix somehow as well, but I need to start making bigger strides with less messing about with the fiddly things!
Coding Challenges And Solutions
Some of the challenges I’ve faced, and (mostly) overcome, so far include:
Cropping a detected face out of the DeepLens video feed in the Lambda Python script. Turns out this is very simple, but it took me a while to figure out.
How to convert the cropped face image to a jpg and write it to disk. Also very simple in retrospect, but I’m a moron.
I thought it would be easy to write the resulting face jpg to AWS S3 from the DeepLens edge device, but this one I just could not figure out due to permission issues. I can write to S3 using the aws cli as the aws_cam user, but so far I’ve not been able to extend those same permissions to the ggc_user account, which seems is what runs the awscam software. I even hard-coded credentials in the creation of my S3 client in the lambda code, but still had permission problems. I had to back-off from hacking on the device out of fear of really screwing something up, however. Best to stay off the DeepLens as much as possible in retrospect.
The only way I was able to get a face image off the DeepLens and into the cloud so far is by converting it to a base64 String, putting into a JSON object, and putting it on the IoT Topic. I worry that all this data transfer is going to cost me an arm-and-a-leg by the end of this thing…
When creating a lambda function to read from the IoT Topic, I kept getting a random error when trying to save it, which made no sense as I was following an AWS Blog Post for how to do the same. Then I found this: https://forums.aws.amazon.com/thread.jspa?messageID=825417&tstart=0. And this is what makes hackathons using new technology so fun! Writing software is really just lots of Google Searches.
And speaking of the Internet of Things (IoT), to-date I’ve thought this was just another marketing buzz word that wasn’t going to pan-out, so to speak. I used to think the same about ‘cloud’ (and still think this about Bitcoin and its ilk). But this DeepLens development challenge is giving me a greater appreciation for IoT and edge computing. In fact, we’ve been talking about the proliferation of internet connected things and the resulting possibilities since Java Jini, and probably before that, but I suspect Python will be its great enabler instead of Java at this point. But I digress…
Baby Steps, But Machine Learning Learning No Where In Sight
So as of today, I am able to leverage the stock face detection model to detect and crop a face out of a live video feed from DeepLens, send it up to the AWS Cloud Lambda IOT Topic Listener, and put it into an S3 Bucket. Next step is to try to figure out how to use the AWS Rekognition service to recognize face images in an image cache.
The Flow Zone
I’ve found listening to music particularly distracting these last few days. However, I find this Horn Solo in Tchaikovsky’s 5th Symphony really soothing and not distracting (but too short). I played this solo in Solo and Ensemble in High School. I’ve been told that french horn players are better kissers…
So far, I’ve learned some painful, hard-fought, lessons in the last two days. I was initially able to register my DeepLens device with AWS Cloud, no problem. The first hiccup I encountered was when I tried to push one of the pre-made models down to the device. They simply would not go, and there are no logs to look at, as that might be too helpful. So, thinking like a DeepLens device myself, I reasoned I probably screwed-up the IAM roles when I tried to register the device (later I learned my assumption was spot-on). To correct the model push problem I was having, I Deregistered the device hoping I could simply go through the Registration process again making sure my IAM Roles were properly configured. And wouldn’t you know the dang wifi on the device stopped working preventing me from logging in to the device to re-register it with the cloud.
The way the DeepLens currently works is that you can only configure it (and upload the certificates it needs to identify itself with your AWS Account) by using it’s on-board wifi and pointing your web browser (on another computer) to http://192.168.0.1. I still can’t get over how odd this is – not sure what Amazon was thinking with this 🙂 . I think it’s odd because my first inclination is to treat the DeepLens like a first-class computer, meaning I have my keyboard, mouse and monitor connected to it. Why would I need to configure it from another computer over wifi? OMG so funny!!
Whither Went My DeepLens Wifi
So the wifi simply would not come on again, as life’s ironies often dictate. So my girlfriend and I went out to Best Buy in 20 degree weather (I bet your girlfriend wouldn’t do that) to buy a USB Hub and a USB-to-Ethernet connector, the idea being that if I could get the device online over ethernet, maybe I could configure this thing that way. Using a hard-wired ethernet connection, my DeepLens was back online, but now with an IP Address of 192.168.1.13. The instructions say to connect to your device console using http://192.168.0.1. Being the contrarian that I am, I tried to connect using http://192.168.1.13 – yeah, no dice. In fact, I could not even find anything running on port 80 of the device at this point. What had I done?!?
AWS guys, I’d totally put an ethernet port in the back of this device.
After poking around a bit, I found the awscam software in /opt/awscam. It looks to me like the DeepLens console is just a nodejs app that is served by some python scripts in the daemon-scripts directory. And wouldn’t you know, those scripts are hard-coded to bind the nodejs app to the wifi device and to run on port 192.168.0.1. I’m dying here. Ok, so I either have to figure out how to modify the daemon python scripts to use the eth0 device and bind to 192.168.1.13, or I have to get the on-board wifi working again.
Luckily, I saw a mention on the AWS forum about a possible Linux Kernel incompatibility with the DeepLens wifi hardware, so I decided to try the path of getting the wifi hardware working again by reverting to an older Linux Kernel, if one even existed – I didn’t know at this point. The following video got me over the hardest piece of solving how to boot an older Ubuntu Kernel:
The GRUB Loader does not display upon reboot in the DeepLense by default, so my first step was to get the GRUB Menu to show:
comment out GRUB_HIDDEN_TIMEOUT
sudo update-grub (or do it as root)
Once the device reboots you will finally see the GRUB menu – fantastic! Select advanced settings, then select the 4.10.17+ kernel. Once rebooted, the on-board wifi should be working again and the little blinky middle light should be happy again. Now you should be back on track to register your device per the AWS instructions. And if you ever need the happy blinking middle wifi light again, the setup pin hole in the back of the camera should work as long as you are running the correct Linux Kernel.
I’m not positive the kernel is the problem, but I am positive these steps worked for me. And how did I get kernel 4.13.0-26-generic installed in the first place? I’m not even sure. I did try to update my device, and maybe that was the start of the problem? I’m not sure.
Anyway, I am now able to download the pre-built Face-Detection Project to my device, as seen here:
At this rate, it’s doubtful I’ll get anything built by the hackathon deadline, but it’s kind of fun messing with the hardware.
This Armin Van Buuren Ibiza set is so tight. Love it, especially around minute 40!
I know next to nothing about Machine Learning. Shoot, I don’t even have a C.S. Degree. But damn the torpedoes, full speed ahead, I’ve committed to completing a project for the AWS DeepLens Hackathon, currently slated to conclude on this February 14th, 2018. Technology is advancing at a break-neck pace and this is one way to try to keep up. Plus, I’ve heard that the first Trillionaires will be minted from the A.I. Industry, so show me the money! I thought it would be cool to blog some of my experience using the DeepLens technology during this hackathon (and I’m actually writing this blog on my DeepLens device).
Actually, on a side note, I’m a little worried about where technology is going these days, especially with such a strong emphasis on A.I. and autonomous machines and all, all driven by profit and power motives, not REAL problems. If we’re not innovating, we’re dying, right? But as Sun Tzu once said, keep your friends close but your (potential) enemies closer.
Confronting the Beast
I went to AWS re:Invent last November in Vegas and somehow managed to get into one of the last DeepLens Workshops of the Conference. Competition for these workshops was, shall we say, fierce! Attending meant I had to miss the re:Play party, but at that point I didn’t really care since the D.J. was not Van Buuren, Garrix or AfroJack. By attending the DeepLens Workshop, I was able to take a free DeepLens computer home, and even received a voucher for $25 worth of AWS Credits to get started.
I was really psyched to get started on the hackathon upon returning home, but I already had a project in progress I had to complete first. Fortunately, I finally completed my Android App (my second Android App ever) and got it released in the Google Play Store (CANDLES Tracker) on 1/11, so I finally have my evenings ‘free’ to devote to this hackathon.
So yesterday, 1/12, I cracked open my DeepLens box and unpacked the device. I realized I needed a new keyboard, mouse and HDMI cable with a micro-HDMI male end. So last night I ordered these things, off of Amazon…of course, and received them in a Prime Shipment by the time I got home from work this evening.
Tonight, 1/13, my DeepLens is all connected and registered with my AWS Account. I imagine it’s not going to take me long to burn through the $25 AWS Credits when I start uploading data to train my models, which I will hopefully get a better feel for this weekend.
Once the camera boots-up, you can log into the OS using the password ‘aws_cam’, which is the same as the username. You can connect to wifi and use Firefox to get on the internet and access your AWS Account from there. Strangely though, the instructions say to connect to the DeepLens Wifi endpoint from another computer and configure it using a browser pointing to http://192.168.0.1. I found this strange as I was already logged into the device, but could not get Firefox to connect to http://localhost to connect to the configuration portal from within the device. But it’s all working now by simply not thinking and following the instructions.
Start deploying some of the Amazon pre-built models to get a better feel for the deploy process and integration possibilities outside of the device.
I’ve recently gotten hooked on this Carl Cox Ibiza set, which is a nice groove for hacking to, I’ve found:
One of my goals this year was to run the Marine Corps Historic Half, on May 21st, 2017, with two of my three kids this year (my youngest has no interest in running 13.1 miles, understandably…). The last time I ran with my oldest two kids was in 2011 when we ran the Marine Corps Irish 10k. Here are some pictures from our 10k run on March 26th, 2011:
We have not run together since 2011, so I wanted all of us to get back out there to tackle bigger and better challenges. We had a great time yesterday at the Marine Corps Historic Half Marathon (13.1 miles) in Fredricksburg, Virginia. My kids and I have come a long way since the last 10k run we did together! My girlfriend joined us on the run.
Our next big goal is to run the Marine Corps Marathon together this October. Oohrah!
Society grows great when old men plant trees whose shade they know they shall never sit in.
I love synchronicity – the Jungian idea that events are “meaningful coincidences” if they occur with no causal relationship yet seem to be meaningfully related. This Spring, I started reading Mike Michalowicz’s book ‘The Pumpkin Plan’. The central idea of his book is that business people should be more inclined to trim away customers to focus solely on their best customers in order to grow them, and their company, to the biggest size possible. Mike likens this business focus on the best customers to a farmer who tries to trim away all pumpkins on a vine to a select one or two in order to grow the biggest pumpkins possible. This Rhode Island farmer’s pumpkin grew to 2,261.5 pounds!! What?!?
Growing is what Spring is all about. For some reason, this Spring in particular has had me focusing inordinately on growth: growing my own vegetables, growing my income, growing my net worth, growing my muscles, growing my cardiovascular strength, growing my family bonds, helping my employer grow. Every day I think about GROWTH. How can I grow more? How can I get bigger? How can I 10x my life??!?! I’m done shrinking!!! I look at the earth – not the World as a whole, but dirt – and biological organisms and how life literally springs forth from it every Spring. No matter what man does to the planet, seemingly, life still springs forth every year. The life force is so strong on earth. Life wants to grow! Life must grow! It can’t be stopped. That’s what this planet does – springs forth life and growth – and humans are no different.
My girlfriend and I started a garden in our back yard a few weeks ago. It was back-breaking work. We got covered in dirt and mud. It rained as we worked. Our backs and hands hurt. I could barely stand upright the next day. It felt awesome. We now have spinach, beans, herbs, bee-balm, tomatoes, potatoes and cucumbers growing. We also have planters of grape vines, black berries and blue berries growing. Despite our lack of farming skills and knowledge, the earth continues to spring forth life. The energy to give life, emerge and grow is unstoppable and everywhere. It’s awesome to think that we humans are products of this energy.
We didn’t stop at a garden in the back yard though. I bought some land down in Southern Virginia this Spring so I could grow even more life! My family and I were down there last weekend (West of the Richmond area) planting Apple, Plum, Cherry and Oak Trees. We also seeded Sun Flowers, Wild Flowers, and some other seeds. If we let the land sit for long enough without intervention, trees and weeds of all sorts would eventually take over the land. I am trying to impose my own growth plan and will on the land instead by determining what life I say will grow there. Why must we grow Fruit and White Oak trees and Sun Flowers, JC? Because I said so, that’s why. I command it to grow!!
Also in the last few weeks, I discovered this guy, CT Fletcher, and how he uses the phrase, ‘I command you to grow!’, to grow his muscles as he lifts weights. He commands his muscles to grow! Why? Because he said so! It’s his ‘Magnificent Obsession’! This is genius! CT has learned to envision the change he wants to affect in his life, and to impose his will over it to make it so. Can I do that too? Can you?
My new mantra when I look at my Bank Account, my Gardens, my Trees, my Relationships, AND my muscles is: ‘I command you to grow!’ Why JC? Because I said so, that’s why!
When I was in 6th grade, I discovered one of my Dad’s records (what the heck is a ‘record’?) called ‘Switched-On Bach’. I gave it a listen and was ABSOLUTELY mesmerized. I had never heard anything like it! The album was full of J.S. Bach music played on a Moog Synthesizer by Wendy Carlos. According to Wikipedia, this album placed in the top 10 on the US Billboard 200 between 1969 and 1972! The combination of old school Classical Music being played on a high tech instrument like the Moog Synthesizer was a blissful combination to me.
Years later I fell in love with the movie, ‘Tron’. The Tron soundtrack was authored and played by my idle from the ‘Switched-On Bach’ days, Wendy Carlos. It was about this time (I was a freshman in High School), that I started to get what an awesome combination computers and music made.
Since then, I’ve tinkered with digital music. I’ve written a few goofy songs. I’ve tried my hand at producing a few songs on Garage Band. I’ve dreamed of becoming a DJ, even talked to a few DJs about how to go about it. But I’ve never pursued this passion much further than that. I’ve been called a ‘fart in a frying pan’ because I chase alot of different dreams. I do need a bit of focus in my life…Anyway, two years ago I took my girlfriend to U Street in DC once to try to get a feel for the DJ scene in Washington DC. The main DJ was Afrika Bambaattaa, one of my faves in High School. We had an awesome time, but the vibe wasn’t really what I was looking for. U Street is no Ushuaia.
Fast forward to 2016. Never mind how old I am now. Not important. This past November, I had the privilege of attending a Martin Garrix performance in Las Vegas with my girlfriend. We were blown away by the richness of the bass, the visual sensations, and the overall experience. The Martin Garrix performance was at the AWS re:Invent 2016 Cloud Conference at the re:Play party. I felt I had been reconnected with my childhood fascination with computers and music. It was SENSATIONAL!!!
Fast forward to today. Where is the EDM scene? Where are the best DJs in the world? Where can we go to hear them and experience the vibe? Turns out one of the best EDM DJ experiences in the world is in Boom, Brussels at Tomorrowland. So guess where we are going this July?!?!?! Headliners currently include most of my favorites, including Martin Garrix, Armin van Buuren, Afrojack, Kaskade, Steve Aoki, etc. I’m hoping that David Guetta, Dmitri Vegas and Like Mike are also there for the Weekend 2 performances – BE THERE OR BE SQUARE!! Tickets have already sold out. We are so stoked!!!!
If you are going to Boom this summer, drop me a line.
Building on the knowledge learned from my previous two blog posts on my following of the Wild Rydes AWS Serverless Computing Tutorials, ( Wild Rydes Part I and Part Deux), I decided to put some of that information to use in my own work at www.nautilustracker.com.
I’ve been working on some mobile apps and a back-end platform supporting my trans-Atlantic Ocean Rowing attempt last year with my girlfriend, Cindy. I’d like to turn some of the things I’ve developed thus far into a Software as a Service (SaaS) for other people to easily use on similar adventures. To that end, I wanted to quickly create a responsive website to put out some information about my future offerings, including the ability to allow interested parties to contact me by providing their email address and a contact message in a simple contact form.
Know Your Limitations. Build On the Shoulders of Giants
> aws s3 sync . s3://www.nautilustracker.com
I had previously registered my Domain Name (nautilustracker.com) with GoDaddy last year. Now I wanted to move the DNS Registrar to AWS. This turned out to be very easy. Once I followed the documented steps to move a domain to AWS, I only had to add an A Record to point to the domain to my S3 Bucket containing website artifacts. I will point this A Record to a CloudFront endpoint soon.
Lipstick On A Pig
Now that the landing page is up, there is a mountain of work to do. The next step is to get email working for my domain using AWS SES so I can use that domain email to register as an organization in the Apple iOS Developer Program.
I took my family to Paris, France for the 2016-2017 New Years Eve celebrations. We were essentially just there for the weekend, but it was a fabulous weekend – one that I will never forget. The sites and food were unforgettable. For New Year’s Eve, I treated my family to an 8 course French Meal and Wine/Champagne Pairing at the Hotel Raphael in Paris very close to the Champs-Elysées. My kids and I were definitely not used to fancy food such as this, but it was a beautiful and tasteful introduction to food as art (as opposed to American style ‘fast food’). Here’s a video of my dessert (thanks Alison ;):
In Paris, we were delighted to see the Eiffel Tower, The Arc De Triomphe, The Palace of Versailles, and to listen to Vivaldi’s Four Seasons, and other beautiful musical pieces, performed by a Chamber Orchestra at the Église de la Madeleine Roman Catholic Church. I left Paris with a profound appreciation for the art and beauty that abounds in this wonderful city, from her cuisine to her architecture, to her music and her people. The French seem to have a genetic disposition toward an aesthetic appreciation of life, which I find hard to come by in the United States. The french term, ‘Joi De Vivre,’ comes to mind when I reflect on our weekend trip to France.
According to Wikipedia: “It ‘can be a joy of conversation, joy of eating, joy of anything one might do… And joie de vivre may be seen as a joy of everything, a comprehensive joy, a philosophy of life, a Weltanschauung…'”
I happened to catch a Netflix Documentary this evening called “Chef’s Table, France,” Season I, which was a story about an amazing French Chef, Alain Passard. As I watched the story unfold about how Chef Passard became a Chef, identifying his career path as early as 14 years old, how he found a mentor and soon bought his mentor’s restaurant, which he named ‘Arpege’, I was completely drawn in by Chef Passard’s sense of life purpose, mastery and pursuit of excellence in his craft. It is not often you find or learn of someone who absolutely loves what they do for a living. I hung on his every word in this documentary and even took notes, hoping to graph some of his sense of aesthetics and Joi de Vivre into my own life and professional career. Here are some of Chef Alain Passard’s quotes and anecdotes I noted from the Netflix Documentary, ‘Chef’s Table, France’:
“When you close your eyes at night, what’s important? You’ve spent the day taking risks. You’ve made some people very happy.”
Chef Passard relates that what you create is just as important as how you create it, which he refers to as ‘Gestures’ or ‘Hand Gestures’. The way you move your hands to create something of value is important and takes hours and years and decades of practice. Chef Passard’s Grandmother was an amazing cook; his mother sewed and his father was a musician. His Grandfather was a sculptor who worked with wood. He learned the importance of hand gestures early in his life and applied them to his craft. He works bread dough like it’s fabric. He sews Duck and Chicken together to create a unique dish. With regards to the hand gesture, he says: “In cuisine, in music, in sculpture, in painting, it’s everything. Either we like the gesture, either we like the hand, or we do not. And this hand, if we want it to be more beautiful, we must work seven hours, eight hours, ten hours in the kitchen every day. This makes the hand more precise, and more elegant.” He goes on to say that a 14 year old does not have the precision of hand that a 30 year old cook has. He says, “I am never happier than when I put my fingers on a new gesture or a new flavor. It feels wonderful.”
“You really become a cook between 40 and 50 years old.”
Can the same not also be said about other professions as well?
When Chef Passard started his restaurant, Arpege, he says that the one and two star ratings came fairly easily, but the three star rating was very difficult to attain. Three Stars is the highest rating for a restaurant. Maintaining three stars is apparently extremely difficult to do, but Chef Passard’s mentality is to pursue higher and higher standards, never stopping or resting upon his current achievements. The search for excellence is never ending, but it’s something he loves. I was struck how there was no mention of the pursuit of money in this documentary, it was purely the pursuit of passion, excellence, and the art of food. In fact, there came a point in Chef Passard’s professional career where he was losing his passion for cooking meat, so he decided to take a year of introspection to find his passion again. He reinvented himself and his restaurant as primarily vegetarian while still maintaining their three star rating. He found a new hand. A new outlook.
“My only ambition is to love what I do more each day. Just the idea of a job well done. No outside projects, needs, or dreams. If this story exists today, it’s because I love my job more than anything.”