Robots and AI and Neural Nets, oh my!

“R2D2! You know better than to trust a strange computer!”

— C3PO

——————————————————

We are living in the future my friends.  The future my childhood promised where artificial intelligence is a real thing and bots are being programmed into human-like bodies.  In case you missed it, check out the press conference with Sophia, the Saudi Arabian robot who was granted citizenship by the government by clicking on this link.

sophia

Sophia is creepy.  Seriously. Just watch her segment on Jimmy Fallon. I prefer my robots to be either indistinguishable from humans (like Janet on the good place, Vicki on Small Wonder, or D.A.R.Y.L) or very clearly a robot (like Johnny 5, C3PO or Bender on Futurama).

In episode 2 of Unsupervised Binge Watching, we discussed these bots and likely how they were programmed to think or retrieve knowledge.  One of the promises was instructions to how we build a bot to communicate.  There will be no deep learning for these little guys, but rather a conversation service, text to speech and speech to text.  So…. Here’s the delivery! Meet TJBot!

tj1

Building TJBot (named for Thomas J Watson) doesn’t take too long.  I created two of them and each took less than an hour.   If you’re going to our THINK conference, catch up with me and we can build together.  Otherwise, catch me at a meetup, or feel free to use the instructions below.  To give credit where credit is due, I must note that TJBot was created by IBM research and the instructions I’m posting below are theirs and available both on GitHub and instructorless. You will need the free version of a few Watson services.  If you think you’re going to want to do a lot with TJBot let me know… if you meet some criteria I may be able to get you even more free services than what we have posted on IBM cloud. Don’t be shy about asking!

With that… happy building and show me what you make!

Build TJBot Out of Cardboard

In this instructable, we guide you through how to obtain an IBM TJBot cardboard and how to fold TJBot from a laser cut cardboard.

If you already have your TJBot cardboard, jump to step 3 for folding instructions.

Step 1: Download the Design Files

TJBot is an open source project for building a little robot powered by IBM Watson. You can download laser cut designs from the TJBot website and laser cut your own TJBot. Three file formats are available: pdf, eps, dxf.

You can also download the 3D printing designs and 3D print your TJBot.
In this instructable, we guide you through how to fold TJBot from a laser cut cardboard.

Step 2: Laser Cut TJBot

The recommend material for laser cutting TJBot is chipboard. Chipboard (same as cereal box material) is thinner and stronger than corrugated cardboard and can be easily cut with a laser cutter to produce perforated edges that are easy to fold.

If you don’t have access to a laser cutter, there are many online vendors who provide laser cutting services. Alternatively, you may be able to find a laser cutter at a local makerspace, along with experts to help you perform the cut.

For the actual laser cut, we recommend using chipboard with dimensions 18″ x 18″ x .080″ (457mm x 457mm x 2mm).

If you are in the UK, instead of chipboard they call it 2mm “craft card”.

Step 3: Folding TJBot (Part 1)

Now that your laser cut is ready, it’s time to assemble TJBot. Folding the chipboard requires patience and your full attention! 🙂

Make sure you set aside enough time for folding TJBot and don’t rush. It’s easy to make a mistake by folding the chipboard in the wrong direction, and reversing the fold will cause the chipboard to break off. Don’t panic if this happens, as you can use scotch tape to repair the damage.

Please note that when folding the chipboard, the following tricks are helpful.

  • Use the edge of a table or a metal ruler to give yourself leverage when folding the seams
  • Don’t overfold – use the smallest amount of pressure needed to make the seams start moving, but stop just when they start to give. If you use too much pressure, you may tear the seam.
  • Plan your folds – as the saying goes, “measure twice, cut once”, make sure you are folding in the right direction before you begin your fold.

Begin by laying the chipboard on a flat table.

tj3

The first step is to fold TJBot’s jaw. The jaw piece is the large one with the four “Fold Down” tabs at the corners. Pop it loose from the larger piece of chipboard.

tj4

Fold down the corners that say “fold down”.

tj5

Next, fold down the rectangular piece connected to the “Fold Down” tabs. This will make a square shape between the rectangular pieces and the “Fold Down” tabs.

tj6

Now, fold down the other two rectangular sides. Fold the inner seam first.

tj7

Then fold the outer seam over the “Fold Down” tabs and punch down the little tabs into the holes to finish the section.

tj8

Congratulations, you’ve assembled TJBot’s jaw! Set it aside for now.

Next up are TJBot’s legs. Pop the two legs loose from the chipboard. The legs have long rectangular pieces with “Fold Up” and “Fold Down” written on them.

tj9

The legs are the most delicate pieces in the set and require patience and focus. Begin by folding the piece that says “Fold Up” in the upward direction. Next, fold down the piece that says “Fold Down”. We have found it to be easier to fold the “Fold Up” piece first and then fold the “Fold Down” piece.

tj10

Now, insert the legs into the jaw. The legs go into the L shaped holes in the jaw until the “Fold Up” piece hits the base.

tj11

Insert the legs from the top of the jaw downward. It may take some force to push the legs through the holes.

tj12

Next, remove the leg brace.

tj13

The brace sits in the two rectangular holes in the legs underneath the jaw. It keeps the legs straight.

tj14

Now, remove the feet and place them before you as shown below.

tj16

Fold the side flaps downward (in the direction of the table), very weakly. The feet will form a triangle shape that attaches to the legs, so over folding may cause them to break.

Insert the feet into the notches in the leg as shown below.

tj17

Congratulations, you can now make TJBot stand on his own! You are now halfway through with the folding. 🙂

Step 4: Folding TJBot (Part 2)

Punch out the three retainers from the main chipboard.

tj18

Begin with the retainer that has the “Fold Down” tabs, and fold down the two tabs.

tjbot19

Slide the retainer downward through the L-shaped holes in the legs. This may require some force, but work slowly and gently to avoid warping or twisting the legs or the retainer.

tjbot20

Servo insertion (optional). If you have a servo, it’s time to insert it on left side hole in jaw.

Next, we will insert the Raspberry Pi. It is mounted face down, with its ports exposed via the three holes in the jaw. Ensure the pins are oriented as shown in the picture below.

tj21

Camera insertion (optional). Remove the camera mounting bracket.

tj22

Slide the two camera side mounts into the notches on the front of the retainer.

tj23

Slide your Pi camera into the mount and then add the top & front reinforcers to keep the camera snug. The picture below shows the camera mount without a camera; please refer to the video to see how the actual Pi camera fits into TJBot.

tj24

The next retainer is the one with the small circle carved into it. Add it to TJBot by sliding it through the L-shaped holes, with the small circle oriented to the right (when TJBot is facing you). Slide the retainer down until it meets with the top of the Raspberry Pi.

tj26

Insert the LED into the center hole of the small retainer and connect the three jumper wires to the LED’s pins. For now, don’t worry about inserting the wires into the Pi; you can hook these up later when following your first LED recipe.

tj27

Push the top retainer with the LED through the L-shaped holes in the legs such that just a tiny portion of the leg sticks out from the hole. Run the wires through the small circle in the middle retainer so they can be connected to the Pi. Note the image below doesn’t show how the jumper wires are routed; this detail can be seen in the video.

tj28

At this point, you don’t need to be worried about wiring the LED. Depending on what you plan to do with your Pi, the wiring might be different. But if you are so keen in getting everything connected, here is a general wiring diagram.

tj29

We’re almost done! The last steps are assembling TJBot’s arm and head.

Step 5: Folding TJBot (Part 3)

Let’s fold TJBot’s head! The first step is to remove the large head piece from the main chipboard.

tj30

Fold down all tabs marked “Fold Down”.

tj31

Fold down the piece with two circles (these are TJBot’s eyes!).

tj32

Fold down the piece with the large hole in it (this is for TJBot’s speaker). Then fold down and over the two remaining sides to complete TJBot’s head. The easiest way to perform this fold is to begin by folding the outer seam (very weakly), and then folding the inner seam (also weakly). Next, fold over the entire piece and lock in the tabs.

tj33

tj34tj35

TJBot’s head is complete! Set it aside for now.

The last step is to fold TJBot’s arm. Remove it from the main chipboard.

tj36

Beginning from the outermost seam, fold everything toward the center rectangular piece (the one with the circle).

tj37

Fold the small square piece with the hole inwards.

tj38

Fold the remaining pieces inwards.

tj39

Fit all tabs into the notches to complete the arm.

All done!

If you have a servo, now is the time to attach the arm to it. Please refer to the video for details. If you don’t have a servo, you can attach the arm to TJBot using tape.

Now that your TJBot is ready, bring him to life with one of his recipes! We suggest beginning with Use your voice to control a light with Watson.

TJBot – Build a Talking Robot With Watson Conversation

This instructable guides you through connecting a Raspberry Pi to Watson conversation services and making a talking robot. You will use (1) Watson Speech to Text to convert your voice to text, (2) Watson Conversation to process the text and calculate a response, and (3) Watson Text to Speech to talk the response back.

Step 1: Parts

Step 2: Prepare Your Pi

If you have used Raspberry Pi before, install Node.js and go to the next step. Otherwise, follow the instructions below to set up your Pi:


Getting Started with Your Pi

Raspberry Pi is similar to a full computer, which means you need a monitor, mouse, and keyboard for it. If you have a TV around, you can connect your Pi to your TV via a HDMI cable. In most of the Pi kits, the SD card is already preloaded with an image of the Raspberry Pi Operating System. You need to put the SD card in the Pi, turn the Pi ON and follow the instructions on screen to complete the installation of the operating system. If you have problems setting up your Pi, you can troubleshoot here.

Here is an easy way to get your Pi connected to wifi.
Install Packages

Open a terminal application on the Pi and execute the following commands to install the latest version of Node.js and npm (Node Package Manager). You need these packages later to run your code.

curl -sL http://ibm.biz/tjbot-bootstrap | sudo sh –

Step 3: Assemble TJBot

Here is a quick video of how to fold the kit. The detailed instructions are available on another instructable: Build TJBot out of Cardboard

Once your TJBot is ready, plug in your USB microphone and the speaker.

tj50.png

Depending on which audio output source you are using with your Pi (HDMI, 3.5mm audio jack, Bluetooth, USB speaker), you may need to set the audio config.

HDMI/ 3.5mm Audio Jack

If you use HDMI or 3.5mm audio jack, you may need to set the audio config. To do this, go to the terminal and open up raspi-config.

sudo raspi-config

This will open up the Raspberry Pi configuration screen:

tj51.png

Select “Advanced Options” and press Enter, then select “Audio” and press Enter. Choose the correct channel for the output audio. If you have connected an external speaker to the audio jack, you should select 3.5mm jack.

tj52.png

USB Speaker

If you have a USB audio, you need to update your /usr/share/alsa/alsa.config to set the USB audio as the default device. Begin with running the following command to make sure your USB is connected and listed there.

lsusb

Next is to detect the card number of your USB audio.

aplay -l

Take a note of the card number associated with your USB Audio. Then go to the alsa.config file to set it as default.

sudo nano /usr/share/alsa/alsa.conf

Look for

defaults.ctl.card 0defaults.pcm.card 0

and update the card number (0 here) to the card number of your USB audio.

Different versions of Raspberry Pi OS may need a different setup. If you have problem with your USB setup, check out this guide to troubleshoot.

Bluetooth Speaker

If you plan to use a Bluetooth speaker, you may need to install the Raspberry Pi Bluetooth software. Here is a good tutorial to set it up: Play audio on a bluetooth speaker with Raspberry Pi 3

Step 4: Download the Sample Code

The source code is available at github. Download or clone the code and execute the following commands from a terminal to install its dependencies..

Here are the instructions for how to clone a repository from github if you have not done that before.

git clone https://github.com/ibmtjbot/tjbot.gitcd tjbot/recipes/conversationnpm install

Step 5: Update Your Credentials

In this step, we help you get API access to three Watson conversation services.

th53

You will use three types of Watson services in this recipe: (1) Watson Speech to Text, (2) Watson Conversation, (3) Watson Text to Speech. You need to copy your credentials for all these services.
Let’s start with creating a Watson Conversation instance (If you don’t have a Bluemix account, you need to create one).

You can leave the default values and select ‘Create’.

tj54

Now we need to copy your credentials from Bluemix. Click on ‘Sevice Credentials’ on the left menu and copy your credentials into clipboard.

tj55

Now make a copy of the default config file.

cp config.default.js config.js

Open config.js file using your favorite text editor and update it with your username and password.

tj56

The Conversation service needs one more ID to configure. It’s called the Workspace ID (exports.conversationWorkspaceId). Don’t worry about it for now. In the next step, we will walk you through creating a conversation workspace.

tj58

You are done with the Conversation credentials! Next stop is Watson Speech to Text. Follow the same process to create a service and copy your credentials.

You may leave the default values for the service & credential names and select ‘Create’.

tj59

Now go to ‘Sevice Credentials’ on the left menu and copy your credentials into clipboard.

tj60

You need to update config.js with your Speech to Text credentials.

tj61.png

The last stop is Watson Text to Speech. You need to do the exact same steps you did with the other two services. You may leave the default values for service and credential name and select ‘create’.

tj62

Copy your credentials and add them to config.js.

tj63

Here is the section of config.js that you need to update:

tj64.png

Congratulations! You survived the credentials section! ::robot-smile::

Step 6: Create a Conversation Flow

You need to train your robot with what to say and when to say it. For that, we use Watson Conversation. Open a browser and go to IBM Watson Conversation.

Log in, and in the the top left menu, select the name of your conversation service. Click on ‘Import’ icon to import the sample conversation.

tj65

In the dialog that opens, click ‘Choose a file’ and select ‘workspace-sample.json’ from the ‘tjbot/recipes/conversation’ folder. Make sure ‘Everything (Intents, Entities, and Dialog)’ is selected.

tj66

After the workspace has been imported, click the dotted menu and select ‘View details’. Copy your Workspace ID as shown.

tj67tj68

Next, update your config.js file with the workspace ID.

tj69

Congratulations, your conversation is ready!

Step 7: Run the Code

Now, you are ready to talk to your TJBot!

Open a terminal and execute the following command:

sudo node conversation.js

Watson conversation uses intents to label the purpose of a sentence. For example when you ask TJBot “Please introduce yourself”, the intent is to make an introduction. You can add your own new intents in the Conversation editor, but for now, we have started you off with two intents:

  • Introduction. You can say phrases such as “Watson, please introduce yourself”, “Watson, who are you”, and “Watson, can you introduce yourself”
  • Joke. You can ask “Watson, please tell me a joke” or “Watson, I would like to hear a joke”.

If you would like to create your own intents, we recommend following this guide.

An attention word is used so TJBot knows you are talking to him. The default attention word is ‘Watson’, but you can change it in conversation.js as follows. Update conversation.js and change tjConfig to add the ‘robot’ section:

// turn on debug logging to the consolevar tjConfig = {    verboseLogging: true,    robot: {        name: ‘Watson’,        gender: ‘male’    }};

You can change the ‘name’ to whatever you would like to call your TJBot. In addition, if you change the gender to ‘female’, TJBot will use a female voice to speak to you!

Enjoy! Don’t forget to share a picture/video of your setup #TJBot! 🙂

Troubleshoot

If you see TJBot’s response on the terminal but don’t hear TJBot talking, there is a good chance that one of these two things has happened: (1) The audio output is directed to a wrong channel (you can fix it from raspi-config), (2) your sound modules are blocked. In that case, go to /etc/modprobe.d/ and remove blacklist-rgb-led.conf

Then run the following command:

sudo update-initramfs -u

Reboot and confirm the “snd” modules are running by executing the command “lsmod”. This should solve the problem.

lsmod

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

Powered by WordPress.com.

Up ↑

%d bloggers like this: