Bobo The Orangutan

One of the questions I frequently got asked was how I am achieving this interaction between Bobo and me. I am not using a fancy facial capture device or special camera. I am only using the built-in laptop webcam and openCV with dlib.

I am not an expert so I don’t think I have a right to explain, but for anyone who’s curious, Computer Vision is technology analyzing the picture and returning information. It can be used by traffic cameras to read a car number or by home security cameras to alarm a suspicious intruder.

Using OpenCV opens the accessibility a lot. It’s cheaper for me and at the same time, anybody can try at their home without any special devices.

Face

There are two systems I had to create just for bobo’s face. One is the Blend shape manager in maya. This helps me build complex blendshape trees in the very intuitive UI. I want to share this online as open source when it’s finished but I am not sure when that would be… (far to go…)

The second system is the face shape manager in Unity. Because an FBX export disconnects the maya rig, this system reconnects the rig automatically in Unity. It’s written in C#.

find all corrective shapes and connect with upper levels.

connect jaw joint with jaw blend shapes, eye joints with eye blend shapes

On facial, it’s still almost start point.

Study About Apes

How different they are?
At the beginning of the project I also didn’t realize how different orangutans and chimpanzees are.

First of all, Chimpanzees live in Africa, and Orangutans live in Asia. Chimps tend to live on the ground but are also good climbers, where Orangutans spend nearly all of their time in the trees. Also, whilst chimps live as a group, orangutans mostly stay alone.

Look at where the thumbs are. chimp- orangutan - siamang. Is this because they are good at climbing trees?

Orangutan’s tree-dwelling lifestyle also gives another hint about them. They are very vulnerable on the ground. Looking at their feet standing on the ground makes me cry for tears because they look so uncomfortable. The toes are too long and their ankle is oriented strangely. This explains why all orangutans on the ground are rolling all the time.

Look at where the thumbs are. chimp- orangutan – siamang. Is this because they are good at climbing trees?

About their mind

My study about the orangutan had to go further than understanding their anatomy because ultimately I had to program bobo’s AI. I had to understand how Bobo would think.

Like we know Jane Goodall discovered Chimpanzees are using tools, Orangutans also use tools. They use sticks to get honey, or they use leaves for umbrellas when it rains. Primatologist Frans De Waal mentioned this interesting case in their research.

Somehow I can hear how she thinks. “Yeah, I look prettier with this hat.” Is that so different from me choosing my clothes?

Another case I found is interesting was the spitting water test. This test showed how orangutans reach an untouchable peanut by raising the water level to a container, even though they’ve never seen the same puzzle before. Each tester came up with their idea based on their experience and knowledge. Most testers completed this mission in 8 min. Honestly, I don’t think I could do better than that.

(Suma, an orangutan at a German zoo, often embellished herself in front of a mirror, such as by putting a leaf of lettuce onto her head like a hat while staring at her reflection. Drawing by Frans De waal)

The more I learn about an orangutan’s mind, the more I think they are similar to us. They see, remember, think, learn.

I started programming bobo’s brain but I still felt something odd about bobo’s behaviour. It was because he looked too mature for his age..

Because I don’t have children and hadn’t had any chance to observe a real baby, I had to rely for online reference for this as well. I watched this baby time lapse video and I realized what I was missing.

The most basic game character AI logic is this:

“Have a goal -> Do it”

It seems way too simple, but think about yourself. Do you do something without any reason? Even to stand up from a chair you need a reason. Whether you want to check out the kitchen, go to the toilet or hang out with your friend at the cafe, we grown-ups want to be as lazy as possible. We don’t want to waste our energy. But babies are not like this. When they are active, they have to emit their energy no matter what because their goal is not doing something, their goal is doing anything. So their actions look unpredictable and illogical. They chase after this and that, get interested and lose interest constantly. They seem more like chaos full of adrenaline. (If I did the same, people would be seriously worried about me…)

Left is the diagram I drew.

Ultimately a baby is a baby. A baby orangutan also wants to get hugged, get attention, to learn and play with others. I can’t believe some orangutan babies still get traded as exotic pets. Some of them get sold to illegal animal fight games. This is not how we should treat babies. Even though some orphans luckily get rescued by organizations, they still have to learn how to survive in wild. That lesson is not going to be same as their own kind could give.

Budi who’s the model of Bobo. He was very sick condition when he got rescued by the organization