Building a "Job Finder" skill for my Amazon Alexa Device

Growth

According to a recent NPR and Edison Research Smart Audio Report, 5% of the 1,620 surveyed respondents owned an Alexa device (Echo or Echo Dot) and 2% owned a Google Home device. Even more revealing was the fact that 42% of Smart Speaker owners said that "their Smart Speakers are essential to their everyday lives". By any stretch, Smart Speakers have reached a significant level of product/market fit and in a little over 2 years Amazon Alexa devices are leading the pack and growing at a rapid pace:

Business Insider: Amazon Echo sales and stats - Dec 2016

Skill

I decided to build my first Alexa skill to better understand the underlying Alexa Skills Kit that powers all Amazon Echo devices to give users this new intuitive voice interface. I was particularly interested in voice-user interface (VUI) design and how easy it would be to go from idea to building and then deploying a new skill. For the actual implementation, I decided to use the new Flask-Ask framework by John Wheeler to build the skill along with using Zappa to deploy my Python code into AWS Lambda. 

I settled on a simple idea for a "job finder" skill that allows users to answer 2 simple questions that Alexa asks them about which city and job role they were interested in. The skill would then return the most relevant job listings that had been posted in the last 24 hours according to the answers given. I imagined users waking up in the morning with their coffee in hand and asking their Alexa "job finder" skill to get the latest updates for their ongoing job search. If the skill responded with some particularly timely and relevant jobs, the user could then decide to immediately open their laptop and apply for those jobs or, if not, the user could simply continue to enjoy their coffee and resume the job search later on in the day.

Implementation

I spent a few evenings coding up the "job finder" skill's Python backend and configuring the interaction model (the intent schema and sample utterances) to get it working locally. I ended up with a single job_finder.py file for the bulk of my application logic which is only 116 lines of code and a template.yaml file that contains a number of the templated launch, answer and re-prompt responses which are required for the skill. All-in-all the learning curve was reasonable and most of the learning came from understanding the new type of interaction model set out by the Alexa Skills Kit.

Demo

As of writing this post the "job finder" skill is being reviewed by Amazon's Alexa's team and therefore is unavailable at present to enable on your own Amazon Echo device (the skill should be certified in a week or less). In the meantime I've recorded a demo using Amazon Alexa Testing Tool called Echosim.io:

Learnings

• I've capped the number of jobs Alexa responds with to a maximum of 5. I think this is a manageable amount but given that Alexa might misunderstand the job role given by the user, it could be frustrating to have to listen to all 5 jobs if they are based on an incorrect interpretation. Therefore I would potentially improve the skill's VUI to either have Alexa repeat the job role specified by the user before all 5 jobs are listed out OR I would pause Alexa after the first job listed and ask the user a yes/no question on whether Alexa should continue or not.

• I've purposefully kept each job listing very brief so it's easy for users to consume while listening but it might be useful for users to get more detailed information. I could improve the skill's VUI by asking a follow-up question at the end of listing the 5 jobs to know whether or not the user wanted more details about any or all of the jobs listed.

• I'm using the AMAZON.ProfessionalType intent slot and this is currently only supported in the US. Therefore when the skill is certified by the Amazon team it will only be available on US based Alexa devices. I've asked about UK support for the AMAZON.ProfessionalType intent slots but have not yet received information about when UK support for it will be available. 

• Analytics are very important for knowing what is and is not working from a user's perspective. I often use tools like Google Analytics for websites and Mixpanel/Amplitude for mobile apps which have been very insightful to better understand user behaviour. For my Alexa skill, I'd like to add in something like Dashbot analytics to get a window into what users are and are not doing with the skill.

• Most importantly I would like to find ways to improve the VUI for subsequent invocations of the "job finder" skill. If a user has already performed a prior job search then I'd like to be able to have a search saved against their user profile and then have the skill look it up and ask the user whether or not it should be re-used. This will reduce the number of questions Alexa asks the user upon each new "job finder" skill invocation. Such functionality can be accomplished by identifying users by their UserId obtained from the session and then querying a database for prior jobs searches to re-use.

---

Update

1. I mentioned implementing analytics within the skill previously. I've since integrated VoiceInsights analytics from VoiceLabs.co to begin receiving data for individual and aggregate user sessions across metrics such as usage, pathing, speech and retention. I'm finding it more valuable compared to Alexa's skill metrics data that are provided by default.
2. My "Job Finder" skill is now live in both the US and UK markets:

Prototyping with Google's Prediction API, Python, SQLite and some JSON APIs

:: Introduction

I've been playing around lately with the car2go API v2.1 as I was curious about how easy it would be to ingest it's vehicle location data over time and then try to accurately predict certain outcomes using machine learning algorithms.

I've had quite a bit of success in the past using Google's Prediction API for spam comment detection but this "spam" problem was using classification values and not regression values. Regression values  would have to be used in this type of geo-coordinates based problem.

The last time I used the Python programming language was when I worked at Toshiba in Edinburgh, UK a number of years ago. I was interested to get back to using the language as I remember it being a really elegant language and it also seemed like a great fit to prototype things out. I was using a Macbook Air, which had Python pre-installed, so this made it pretty easy to get up and running. I knew I'd need to query the API, parse out the necessary values and store these to a datastore of some kind. I opted to use Mac OS X's built in SQLite database at it integrated well with Python and it provided a great interface from the command line to test out various SQL queries.

:: Inserting Records into SQLite with Python

The following Python program illustrates how to connect to a SQLite database called car2go.db and insert a single data record into the Vehicles table (Note that the Vehicles table schema needs to be pre-defined and created before the code will execute successfully):

#!/usr/bin/python
# -*- coding: utf-8 -*-

import sqlite3 as sqlite
import sys

con = None

try:
    con = sqlite.connect('car2go.db')

    cur = con.cursor()
    
    cur.execute("INSERT INTO Vehicles VALUES(<INSERT_VALUES_BASED_ON_SCHEMA>)")
    
except sqlite.Error, e:
    
    if con:
        con.rollback()
        
    print "Error %s:" % e.args[0]
    sys.exit(1)
    
finally:
    
    if con:
        con.close()

As you can see this code is really lightweight and easy to get up and running when you have a machine with Python installed. 

:: Requesting API Data and Parsing Values with Python

The next step was to query the JSON based API and parse out any necessary vehicle values:

#!/usr/bin/python
# -*- coding: utf-8 -*-

import json
import urllib2
import sys

# Note that you'll need to replace <SECRET_KEY> & <LOCATION> with the appropriate values
car2goVehiclesApi = 'https://www.car2go.com/api/v2.1/vehicles&oauth_consumer_key=<SECRET_KEY>&loc=<LOCATION>&format=json'
    
# request vehicle data in json format
webReq = urllib2.urlopen(car2goVehiclesApi)
vehiclesJson = json.load(webReq)

The vehiclesJson object now has the JSON data from the web request and can be queried directly given the appropriate keys/indexes. And example to output the address of the first vehicle is:

print vehiclesJson['placemarks'][0]['address']

:: Outputting SQLite Query Data to CSV

Once these basics where done I then had the foundation to expand upon it further. I eventually extracted all the necessary JSON vehicle data from the web request and inserted it into my Vehicles table in my car2go.db SQLite database. From there I crafted the specific SQL query I wanted and output that query data to a CSV file using the following sqlite3 command syntax (simply run the SQL query after setting these values):

.headers off

.output vehicleData.txt

.mode csv

:: Using the Google Prediction API

Setting up your Google Prediction API the first time can be slightly tricky so be careful to follow Google's directions carefully. I won't go into these prerequisite details here as Google has already done a great job explaining them here (and will most likely keep them up-to-date in the future). 

After setting everything up the vehicleData.txt file was then uploaded to one of my Google Cloud Storage buckets where it could be queried directly by Google's Prediction API. Note that depending on the data you upload you may need to wrap the strings in double quotes according to Google's specified training data format. After uploading, the first task was to train my model which can take anywhere from a few seconds to a few minutes. Once this is complete (querying for the status will inform you when its complete) you can begin to ask for regression predictions from the predict HTTP request. Note that depending on what you are predicting the outputValue or the outputMulti[].score values can be retrieved from the JSON response and used to interpret your intended outcome.

(Note that I was using Google's Prediction API v1.5 for this prototype).

:: Conclusion

I'd highly recommend giving the Python language a try (if you haven't already) for your next prototype project. Integrating it with a lightweight database, JSON web requests and Google's Prediction API was pretty easy and my overall impressions with the language and available libraries are that its still a pleasure to work with.