[Music] hi and welcome to how to make better products with Android things I'm Kristen and I'm UX lead for Android things my name is Michael I'm the UX design manager for Android things you can think about this talk as product design 101 for people who may not be designers but if you are a designer we got you covered we'll be covering hardware prototyping and the possibilities of what you can create using Android things in this talk we'll cover how you can accelerate the prototyping and product creation process using Android things we'll talk about a design framework that you can use starting today to help you think about who your users are and how they can play a more prominent role in crafting the products that you're creating and we'll also talk about a concept project called Lantern to demonstrate how we've applied the design framework to use Android rings to create better products thanks Michael so we know that hardware design is a long and difficult process it can take anywhere from two to five years to bring a product from ideation all the way up to product so you start with ideation and then you move on to the prototype phase and then you choose your hardware and then you design your software and get that all coated up and then you send it to a factory you finally get done on the store shelf and then you cycle back and you have to go through updates so we live in a world that's rapidly changing and technology can change right in the middle of your production process so how can the design process keep up that's one of the main reasons that we created Android things it's made for a world that's rapidly changing and enables people to be part of the creation process from ideation all the way through maintenance so at the heart of Android things there's something called a sum or a system on module ok and this is also called a carrier board the song can be used for prototyping and it can be also placed on a custom board so you can snap this song off and use it on your own custom PCB board ok and this carrier board everything that surrounds this carrier board is an accessory so everything from the Ethernet to power to the headphone jack over here this is a powerful tool for prototyping because you already have a lot of tools that you need to get connected and of course if you need a different peripheral you can easily connect it using traditional methods like pins of breadboard and resistors so Android Things offers a number of tools for you to get started easily there is the kit which has an imx7 developer board a touchscreen and a stand so for those of you who have gotten your kit already or maybe been to some of the code labs the kit assembles into a useful stand that you can use to prototype right on your desk in addition to the kit we offer the Android things toolkit app to help you get on to Wi-Fi really easily one of the pain points that we heard from developers in the code labs droidcon for example was provisioning the devices on to the Wi-Fi network was difficult so with the toolkit app you can get it on to Wi-Fi but in a breeze it'll also step you through the process of making sure that your hardware is connected correctly and with recent updates to the toolkit app we also have some samples that you can load from the app onto your device to see some of the powerful things that Android things can do like running the tensorflow demonstrations before you get into Android studio in addition to the toolkit app and the hardware kit we also recently updated the Android things dot with Google comm community hub so now we offer code snippets samples drivers and projects from the community so if you do build something cool you can submit it and we'll feature it on the site we also recently updated the site to include driver submission so that if you do write a cool driver and you do want to submit it for other people to use we can have that on the site as well if you haven't already gotten your kit head over to the i/o dome and they'll give you information about how to get one thanks so Android things provides an end-to-end solution it offers tools from prototype to production Michael mentioned the psalm makes hardware selection easier by offering modular hardware solutions so you can use the same TAS on for prototyping as you do production for prototyping the kit offers peripherals such as displays a camera a rainbow hat for sensor input and interface output and also an antenna to connect the device to Wi-Fi an app also makes this easier as Michael mentioned to assemble your hardware and it also helps you get familiar with your carrier board and it helps you connect to Wi-Fi and finally when you're ready for production the developer console can help you create builds configure your firmware and release those builds to devices great so we wanted to share with you a concept called Lantern and we're using this as a demonstration to help you understand how we were able to use Android things to bring products to life so Lantern is not a Google product but it's a project that we worked on with the Nord group that creates augmented reality anywhere around you and so you can see this as an easy an easy to understand example of how we were exploring creativity through prototyping and again we wanted to show you the possibilities of what you can create what we can create or the possibilities of what can be created with Android things so at its heart Lantern is obviously a lamp but it's a lamp that enables you to create augmented reality anywhere around you and it's you it's created using off-the-shelf parts and we thought that was really important because we wanted to make sure that as a recipe that you could potentially build one on your own so what is augmented reality you may have heard this term AR there's the AR kit but how can we create this sense of augmented reality using Android things so using Lantern and Android rings we wanted to project onto everyday objects interesting pieces of information and content that may be trapped inside the phone or on the web but that may enhance the world around us so say for example here the currently playing calf song we're projecting it on to a speaker and none of this was done using After Effects this is all using the projection system in the prototype that we created another example of how we Rock menteng everyday objects is in this example a clock so we're using Google Calendar and a wall clock with Lantern to project the calendar information around the clock and again this is all real we shot this in the studio using the projected land using Lantern on as an exploration here we felt like it looked particularly good because it was on this nice curved round surface so it gave this ticker ticker tape kind of look but we're excited about these possibilities and that's why we wanted to create this to share with you to demonstrate not only our design process which we'll get into but also to give it to the community to see what you guys may want to create with it and so what is Lantern made of I mentioned before that it was created using off-the-shelf parts so there's a lamp inside is a laser projector an accelerometer a 3d printed housing and Raspberry Pi running Android things it's important to recognize that there are two pieces of hardware that Android things - boards excuse me that Android things runs on it'll run on a Raspberry Pi and it also run on the imx7 boards that are in that are in the kit the Raspberry Pi is is a little bit more prevalent at this point in the maker community so we felt like building it on that platform with the HDMI output was going to be better for this case because we could connect it directly to the laser projector and so once it's assembled it looks like this you may have seen it over in the IOT dome we have one running over there as an example and we really believe that this is only now possible because of the democratization of design and hardware and prototyping and access to these kinds of tools that we're talking about today so it was really difficult in the past to say print a 3d form like this and assemble it into a hardware shell because hardware was expensive 3d prototyping tools were inaccessible and tools like Android things were not easy really readily available for you to access to create new hardware prototypes and so Lantern can also be assigned content to its its particular context so it's aware of its orientation so using that accelerometer in Lantern we can change its base position and then project different content onto different surfaces so say we wanted to project a star chart on the ceiling or in the examples that we saw earlier you could see the calendar information projected onto the wall this code is available today at github comm slash nord project sash lantern if you want to check it out download the source code and build your own so we didn't set out to create Lantern so where did Lantern come from you can see a number of sketches that we created and if your designers you may be familiar with AI da ting through sketching but we have an inclination that projected systems would be interesting when we started prototyping using Android things but we use design to turn idea into a real thing and today we want to share with you the process that we went through and the frameworks that we used to create amazing products so you can - thank you so as Michael mentioned design helps create better better products how many of you with a raise of hand have created something that was used by another person go ahead and raise your hands oh that's awesome so when we design things you might know that we use common principles to ground our work a lot of the things that we design can be carried over from software into hardware from a banking app for instance to a theater app there's another instance so we use these processes and forend principles to ground our work but then we use the design process to move forward as well and then we continue iterating with Android things we're taking some of the software UI design concepts and applying them to hardware so what is design design is the creation of tools for people in a context to help them achieve a goal if any of you are familiar with the development process this is very similar to a user story so for example as a dog owner I want to I want to connect a dog feeder so that I can feed my dog from work or as a person who orders pizza I ordered pizza all the time I want visibility into the delivery route so I know when my pizza will arrive or something like a simple story as a night I want a stronger sword so that I can defeat the dragon now taking the knight example we framed out that the knight wants a stronger sword this is a tool that the knight uses to achieve his goal of defeating the dragon we call this tool an interface so people people typically think of a user interface is a touchscreen or a mobile phone or a tablet because you can tap on the screen and things happen right it's magic and while this is true user interface is much broader than that like in the example I used before the user interface is the sword but we can see here from this slide that a button can be used to build upon an interface to create a joystick which can be used to create a game that shoots down aliens from the sky and then there could be feedback on top of that where there's leds inside of a breadboard that light up when you push the button right so all of these are is an example of a user interface but one of the most simple user interfaces that we use to design things as simple as a piece of paper and a pen and admittedly a design is iterative and sometimes it feels like this right like the hamster in the wheel that's okay the truth is is that you're never done teams need to collectively learn through the experience of observation and iteration that being said Android things enables you to iterate faster by allowing you to work through many design issues by using the design kit as a base for your prototype phase and allowing for early over-the-air updates using the developer console so design is a process that can be used to create better products for everyone and we know that design is agnostic of medium time trend or technology company and we believe it's important to think about design as a process that's agnostic of these things so no matter what changes you have the right tools to apply that process when whatever problem you're working on so we talked about design being in the context of people are people in the context of goals but how is it done and what does it look like so each milestone outlined here planning prototyping getting feedback in iteration needs to be vetted right and using this iterative framework we can enable this to help us make less mistakes produce better products and have a cheaper production process along the way because we learn earlier how the product needs to take shape as its evolving this will truly help you make decisions sooner so thinking about planning planning takes the shape of many forms first we aim to create baseline understandings of needs that may exist for our users so we may begin by talking to people about pain points or getting inspiration from places from pain points that we have ourselves we may look at competitive products to think about how they are solving specific problems and how we may want to do things differently if you're familiar with design for software systems you may be familiar with creating user personas to get an idea of how you can gain empathy into the mind of somebody who's actually using your product we also create things like wireframes storyboards to begin to tell the story of how we seeing the products use unfolding over time we then create something right so based on what we know and the hypothesis of how something should work we begin by creating medium fidelity designs so we solved this really preliminary sketches right we may create something like a video simulation to think about how it might look and feel before it actually works we might make something on a breadboard to get an idea of what it functionally might do or what some of the key characteristics of that functionality might be like and then finally getting feedback so it's important to get feedback in this cycle because we need to understand how people are actually using things so qualitative feedback understanding the users perception of how they feel about a particular feature or what you're proposing quantitative research we can be used to gain data to understand how specific features are you are being used or not used internal feedback we're constantly sharing projects with each other internally to get feedback from other people it's really helpful to get an objective eye on something so that they can so that that person can point out something that you may not have seen and gorilla research I'm showing your prototype to somebody who may not be familiar with the project can give you tremendous insight because then you can have an objective set of eyes on features that you may be creating from somebody who may not be familiar with the project's perform and finally in later stages you may employ something like a lab study more formally to ask participants to use your prototype to see in a controlled environment side by side what different variations may be like and we're not saying that you need to do all of these things along the way so for example we may create a rough storyboard which can then translate to a click-through and then gain internal feedback and iterate on that cycle or for example we may do a bit of competitive analysis or define some must-have requirements and then create a looks like feels like prototype and then perform some gorilla research with somebody who's not familiar with the product we're creating so something else to mention is that hardware isn't hard it's just different you need to consider what parts that you need to make what you want you also need to think about designing a system possibly with non customized parts sometimes it's like Jenga one requirement can actually affect another and you also need to think about future proofing how much space do you need to allocate for user data for example if you're designing a camera this might be important for users and finally you also need to think about form factor how will how will all of the parts that you want to fit into your design fit into a form factor that's delightful for your users one other thing to consider is your interaction design so when you click when you connect the product to the internet do you make a companion app for that how do you make sure that the notifications can be seen from a reasonable view how do you make it accessible take something as easy as a software update do you tell the user at 8:00 a.m. when people are actually using your product or do you wait until 2:00 a.m. when people might not be using your product and they might be asleep also what if there's no screen something else to consider for the land of IOT so we can apply our design process using Android things to help us iterate faster and get back feet get that feedback sooner make better informed decisions for these questions and also ultimately help us design better products and one thing to note if you're a software designer something that you're really familiar with is the undo button in Hardware you don't have an undo button it's really difficult to rollback or release sorry it's really easy to roll back and release when you're a software engineer as well but when you're designing for hardware it's a lot more permanent so you need to have a strong iteration cycle because each time that you move forward in your design cycle the more expensive it gets to move backwards for example in 1966 NASA's budget was over four percent of u.s. spending and the undertaking was mammoth now for example we can shoot 3d printers into space a simple technology up there update it from your laptop and reuse the Rockets all at a fraction of the price we can shoot cars into space simply because it's fun and because we can one other thing to note here is the design process so with the Mercury Gemini and Apollo missions they were all built to get us to the moon mercury put a man into space Gemini to extend this to extend the capsule stay from hours into days and the Apollo to get us to the moon these were some pretty big iteration cycles but you know they were really onto something you may also notice that even today some hardware design is actually based on a waterfall process it requires everything to be perfect and is expensive and end in cycles so for example you start with your requirements you hand those off once those are done to the designers who finish the designs and then they hand that off in to the engineers for implementation once the implementation is finished off then they hand it off to the QA engineers for verification and then you cycle back through a maintenance cycle and you do that over and over again and if you make a mistake or you decide that you want to change if you're late in the process you have to go all the way back up to the requirements section so on the right you'll see something that looks a little bit more like the process that Michael was talking about only except we have a few more milestones sprinkled in there so each one of these is an iteration cycle you plan prototype feedback and iterate and you keep continuing to do that throughout your process iteration is flexible and it helps keep cost contained and the users included in the process in the past there were also very specific roles that contribute to the creation of a project including fabrication engineers now with the access to tools like on-demand 3d printing product creation has become more democratized as Michael mentioned before anyone can print 3d parts designers can code visualizations now with the ease of prototyping tools app developers can also apply their skills to hardware and Android things makes that easy to do so specifically we're dealing with software in the past it was also labor-intensive to create and release a build and it was difficult to set up testing environments for those devices as well with Android things we've introduced the developer console so that users can update their app in Android studio open up the dev console and create and Bill create a build and release it all in a few easy steps so now that we've talked a little bit about design and process we want to bring this back to the Lantern project that I mentioned earlier so how are we able to use the process of planning prototyping getting feedback and iterating to help us improve this concept we were able to think about people context and goals and then apply this to the simple idea so as a designer sitting at my desk I'd like to bring my room to life through projections simple idea that connects people context and goals we started by sketching but what happened next we began to create a looks like feels like prototype so before we were even assembling Hardware or putting together the housing we started to think about what would a time rendering look like as a projection if it was sitting next to moon or what would I want to project onto the desk in front of me as I was typing would it be some Bitcoin information or a price of something else or how I'm doing in a certain game it looks like feels like prototype can help you understand how something how something exists in its end state without actually having to get there through the full creation process so again we went back to sketching to think about how the housing may come together if we're thinking about a lamp and thinking about the parts that needed to go in it well we as I mentioned earlier we'd need the Raspberry Pi board we need some sort of projector and I'd have to start to fit inside of this specific shape this was one of the first prototypes that we created using foam core in some of the parts so once we knew the parts that we needed we began to put them together and assemble them not into a 3d print right away but just using foam creating slices cutting them out and making the form so that we knew that they would fit into the lamb and before we put everything together we started to prototype what some of these content simulations may look like functionally so on the Left what you're seeing is an accelerometer tests showing how we could change content based on orientation and again it's really simple we had the projector we had the Raspberry Pi connected to an accelerometer running Android things and we just said can we show you the direction or show us the direction of which way the object is pointing so does it know if it's pointing up down or sideways and then on the right hand side these were some of the initial tests that we did looking at how we could get the currently playing song off of the Wi-Fi when you're casting to a nearby device and just projecting it onto a notepad to see is that even possible using the hardware that we think we want to use looking at physical prototyping here's Joe with the first assembled prototype looking at the accelerometer changing the content based on the orientation of the physical prototype and here flips up at the ceiling then you can see the content start to come to life so again we built these pieces up individually and then started to put them together into their final form so one tool that I wanted to mention that the team found highly beneficial in creating this project was processing and so if you're a designer you may already be familiar with processing as a lightweight IDE that enables you to quickly create visualizations for designers like me it helps me because I can create visualizations independently and there's a really nice library that was really recently released from the processing foundation called processing for Android and what that enables you to do is write processing and then test that in on an Android device you can test that on your phone you can test it on your wear device but you can also use that on Android things which is really nice because you can work on a visualization and then that onto your Hardware independently what this enabled us to do was work with the visualizations get them to a place where we wanted them and then integrate them into the hardware I'm a few gotchas when you're moving from processing you can export an Android studio project however you'll need to upgrade the minimum SDK version in the Gradle file in order for you to work with Android things so Android things requires a slightly higher Android build number and once you do that you'll be able to connect your processing visualizations to hardware to manipulate them through the standard Android things GPIO inputs and so thinking about our iterative process how did we go about getting feedback on Lantern so first we started asking team members around us to use it we couldn't really go out to the public because this was a private thing that we were working on but we were able to find other people inside of Google who are not familiar with the project team members such as the ml team on Android things who thought it was pretty interesting but one thing that we had really kind of pushed away from was the idea of integrating in your interactivity into our MVP or our first iteration of the project we were specifically adverse to incorporating interactivity because we wanted to prove that we could project content in different ways interactivity adds another layer of complexity so again we wanted to simplify this into its basic form so that we could prove that the independent on interactions were working however when we shared the prototype with the ml team they were really keen on integrating a camera and so because we were working with Android things and working with 3d printing we were able to make some modifications integrate the camera and the Norden team happened to be working in London and the Mountain View team was over here and it was really an interesting story of how this evolved because again never before have you had access to tools like this like 3d printing from the web we were able to actually build prototypes into physical locations collaborate on them and build them up simultaneously which is really cool and again Android things made it really easy to integrate new hardware like this and connect it to the visualization pieces in a snap and so with the camera in place we now had the possibility of a greater range of interactivity and interactive input and so this led us to creating quick-draw pen and paper addition so for those of you who are at i/o last year you may have seen quick-draw which is a creative labs project from Google that prompts you with a word and then as you draw it on a tablet on the web or on your phone it starts to guess what you're drawing and we thought that would be really cool to do in the physical world now that we had this projected AR system and so what we did is we did just that right so previously it was limited to screen based inputs but we thought wouldn't it be cool if we could use just a pen paper capture that input feed that into the quick-draw engine and then have an interactive game that you could use in the physical world this is an example of the demonstration that we have set up over in the IOT dome if you have the visited the dome you can check it out and you can try it out for yourselves we've got a lantern set up and it works it works great so you're prompted with a word and as you start to draw with the pen and the paper it starts to guess on this projected interface in front of you so again thinking about this as a mixed reality surface and there you go so now we've gone one through one cycle right we've planned we've prototype in and we've got feedback and we've iterated on that cycle we made some improvements and so we're done right we're ready to ship the project no no no we're not going to ship our product yet so what we're going to do is start the production process and so what this means is we're going to move forward maybe mass producing something like Lantern so Android things has many prototyping tools to get you started after you've completed your proof of concept and the initial prototypes everything is working and you know what you want it to look like and you know you want to move over to a custom board this is easy because of the somme like I mentioned before you shouldn't have to redo all of the work that you did in the prototyping phase just because you're using the same architecture to create your products so now let's say that I want to mass-produce lantern right and it's part of the feedback process I wanted to learn more about factory production and bring up processes so I visit a few factories in China as it turns out they have a process too and it Maps well to the overall product design process when you mass-produce a product the factory will build out a line with stations each station is staffed with people who put together the product if I want millions of products or millions of lanterns the factory might automate that process and make some really cool robots and automate things with conveyor belts and those robots during the whole process though it's good for you as a product designer to continue iterating on your product but probably leaning more towards software instead of hardware changes and I'll show you why so you've created 10 units perhaps in your prototyping phase right and then you send those off to the factory and there they're going to run everything through something called a validation test and you start out with engineering validation tests so you send your prototypes to the factory and they'll say back maybe a hundred units and during that time you need to make sure that everything works with the materials that you've selected they're probably going to be using soft tools possibly hard tools in this stage to create all of the forms for your your industrial design and then after that after you've verified everything then you send any feedback back to the factory and you move on to design validation test this is where you move on to stainless tooling all of the stations are set up and they're staffed with people and at this point the product design team should be using Android things console to update software and update all of the testing channels to help make your iteration cycles go better in the mean time you should also be doing user testing throughout all of these phases then when you have feedback for the factory and you send things back then you move on into the product validation test this phase makes sure that everything is moving as fast as possible this is more for the factory than for you they'll send you back around a thousand units and they're you should be continuing to use the console to run and test metrics on these devices and finally for mass production you should be using the developer console to send any zero updates if needed and gather more metrics on these devices so as you can see here as you move through the process they should be sending devices back to you the whole time and you should be testing those devices giving feedback extensively through QA and one other note from a design perspective as you move forward through this framework your solutions might need to get more creative more MacGyver II and lean more towards software solutions because it might be too expensive or too late to go backwards and start over again if you're working with a product team so some of the tools that you can use to make this all go a lot smoother is the developer console so we've included something new called the app library this allows you to add apps to you can this allows you to add apps to the app library and use them on multiple projects so if you have multiple lanterns for example you can write one apk upload it to the App library and then you can use it throughout each of your different products when you're ready to test over-the-air updates you can create releases and channels and so sorry when you're ready to build you can actually go through and use the you can use the build settings and then you can create a build individually and run through each of the steps to set all of the firmware and software for your device and then you create a build very seamlessly and when you're ready to test over-the-air updates you can create releases and channels you can push those releases to channels so you can test your software in different groups for detailed testing so for example here I'm creating custom channel I'm creating an update and I'm going to push these updates to my devices or my fleets of devices wirelessly and finally after you release you can gather metrics to help you gather quantitative data on your products to help you make better decisions about what you need to do next so for example in Lantern version 0.1 you have maybe a set set of features that you want to use and then you want to move on in a quick draw you can do so fairly easily by using the build tool and then you can check in metrics to see if it's doing better and once your device is in the market the feedback doesn't stop there so you should be gathering feedback doing user testing and checking everything as your device is in the market through any updates that you've created continue doing user testing continued doing gorilla testing anything that's in that research column you should be continuing to do to help you create updates for your products and also help you define the next iteration cycles through your thing and that's the process so we've taken you through ideation prototyping with Android things kits making final Hardware selections creating builds and hardware bring up the factory process all the way to the end product with maintenance and updates running through design iterations each step of the way thanks Kristen and so we've talked about today how design is people in context with goals and it's a process we can plan prototype feedback and iterate on that process in order to help those people in those contexts achieve those goals and now it's easier than ever to participate in the creation of physical products then it has been ever good for Android things offers tools to help accelerate the prototyping process that you can use today to kick-start your ideas processing enables the fast experience experimentation of rich graphics and when combined with hardware enables you to make really powerful experiences all of these techniques really enable you to make better connected products for everyone using Android things you can get started today at experiments dot with google.com slash Lantern you can check out some of the samples projects and more at Android things with google.com if you picked up a kid today there's lots of sample code over there for you to try out if you haven't gotten a kit you can get one over at the IOT dome a big thank you to the Nord group Ben Joe and Mike for helping out with this project ding on the Android things UX team for managing all of the intercontinental prototyping that was required for this and Chloe on the Android things ml team for making all of the ml magic come to life please check it out we'd love to see you over there thank you guys for coming and if you have feedback we do want to hear from you and you can do so right here [Music]