Home Assistant

Home assistant is an smart appliance control system you can get in different forms, as a service you install on a linux box, docker container or a Raspberry Pi distribution. I got it this last way, just simply burn a microSD card following the well-written instructions and pop it in a spare Raspberry Pi.

After the system boots you can set up and use hassio through a web interface, the UI is generally pretty intuituive and easy to use. Hassio automatically detects some of the smart devices available on the network and offers to integrate them, letting you take control of your aplliances in a centralised way.

ESPHome

My utsund hack was a homebre solution, there was no way I could connect it to Hassio without reading its documentation, developing a firmware for the NodeMCU that controls it and setting everything up. Luckily for me a bunch of other people went this way before and created the ESPHome project.

ESPHome is a framework that let's you configure an ESP-based device (like the NodeMCU) through YAML files and generates a firmware image based on that configuration. The process is totally automated, the ESPHome CLI tool, downloads its toolchain and dependencies, converts the YAML code to whatever intermediate language it uses and finally compiles a firmware that it burns on the NodeMCU. It even offers you to do over-the-air updates!

The documentation is pretty extensive, but the organization lacking. Finding a way to reproduce what I already had donde for the Utsund string light was a bit cumbersome, but once I found the right documentation pages setting the YAML file up was a breeze.

esphome:
  name: ikealights
  platform: ESP8266
  board: nodemcuv2

# Enable logging
logger:

# Enable Home Assistant API
api:
  password: "super-secret-password"

ota:
  password: "even-more-secret-password"

wifi:
  ssid: "An IoT network"
  password: "thenetworkspassword"

  manual_ip:
    static_ip: 192.168.50.100
    gateway: 192.168.50.1
    subnet: 255.255.255.0

switch:
  - platform: gpio
    pin: D2
    name: "Ikea lights"

I had to define the pin where I had connected the MOSFET which controls the power flow and set up the networking. And that's it! When the NodeMCU connected to my WiFi network hassio offered to add it to its system. From there on I had a switch on the hassio interface to turn on or off the lights from my PC or smartphone!

Automation with hassio

Having integrated the utsund light string with hassio I couldn't leave it at that, hassio offers incredible functionality to make your life simpler. One of those features is automation. Automation routines let you link actions to triggers.

In my case I created an automation to turn on the light string when the sun sets, and turns them off at certain fixed time at night. I also tried out an automation to turn the strip on or off by reading an NFC tag, pretty useful if you want to avoid opening the app and pressing the button yourself.

I wanted to get my feet wet with a similar project, but instead of improving on an already connected vibrator I asked myself if I could transform a dumb vibrator to make it interactive and more versatile. There is no shortage of cheap devices on the market which one can only turn on or off. Could they also be improved?

Establishing a control interface

The vibrator I had lying around at home was a cheap, dumb on/off bullet vibrator. You'll probably find those in any basic sex toy, and are also sold on their own. Some of the higher end devices offer wider control options, like different vibration intensities or patterns.

As with many of my hacking projects I wanted to alter the device physically as less as possible. Given this condition, the obvious way to start with such a device would be to be able to control the power delivery. The chosen vibrator works with a AAA 1,5V battery which is stored inside it. The lid or cover which secures the battery has an embedded on/off switch.

The positive terminal can be found deep inside the device and the negative one is a piece of metal that runs up to the exit through a metal contact. Reaching the positive one seemed like quite of a challenge, but a 5,25 audio jack fit perfectly and made contact with its tip. The jack I found at home was an adapter, so I connected and old earbuds cable, peeled of the plastic and attached a dupont connector to it. The negative contact was easier to access, I simply lodged a cable between the audio adapter and the existing metal contact.

This way I achieved power delivery control. Form here on I connected the vibrator to an Arduino UNO through a MOSFET. The vibrator consumes a higher amperage than the Arduino is capable of delivering, therefore this transistor is needed. A relay could also be used, but would be pretty cumbersome to have, regarding the future minification of the control system.

Ways to control the vib

Having established a way to control the vibrator, even with some leeway to set the intensity I thought out different ways to adjust its power.

Distance through ultrasound

The first one was using an ultrasound sensor to measure distance, for example from a hand to the device, like scanlime wrote in her blogpost. This way the vibrator could be used like a theremin!

The experiment went quite well, but regarding a more realistic implementation, the sensor would need to be calibrated to detect movements in a smaller range near the sensor. It wouldn't be very practical to stretch your arm (or leg?) a meter into the air to make the device go faster. A non-linear mapping between distance and power would also be advisable.

Temperature

A vibrator belongs in a hot setting, does it not? Why not try to control its buzzing through temperature? I reckon that for this to work in a final product the thermometer would need to be somehow embedded in the vibrator and be able to quickly read temperatures in the range corresponding to the human skin.

Sadly, I didn't have such a thermometer. The only sensor available in my drawers was made to measure temperatures in the range from 0 to 400ºC (it was a 3D printer extruder thermistor). But it would suffice to set up a demo. The main obstacle I faced with this sensor was correctly mapping the temperature values to sensible power levels.

In the following video you can see the thermistor being drawn near a soldering iron at 300ºC and some ice.

Inclination

Going the Lelo sensemotion way, using inclination to set the vibration power would also be easily implemented. Using a GY-521 or MPU6050 accelerometer/gyroscope one could measure the angle of the vibrator and make it buzz accordingly.

Although this doesn't sound like a very practical way to control it if the sensor is embedded in the device. Having the accelerometer in a separated in its own device making it work as a remote control, would be more useful.

One could also detect sudden or heavy movements with such sensor. Or maybe even detect a movement pattern with some clever post-processing on the raw data. This could be applied to gamify the vibrator use and reward the user when achieving a series of movements shown in an app, for example.

Music

Last but not least, I wanted to achieve buzzing control through music, but it ended up being more difficult than I anticipated.

I tried to reuse the ultrasound sensor for this use case, but as expected, it didn't really pick up the frequencies used by music. I then got a KY-038 sound detection sensor, but it also didn't work for music. Even though it could see it faintly detected the sound of music, it couldn't really be separated from the background noise baseline.

In the previous image I plotted the raw values the KY-038 sensor picked up during a few seconds. In the middle portion of the graphic you can see some seconds in which I played some music. In the image it may be pretty clear when it happened, but the differential between the values is too small to build a system that detects music reliably. And it even isn't able to pick up the music's rhythm.

After a bit of research I saw some people connected an audio jack directly to the microcontrollers analog input, which seemed like a pretty good idea, but I imagine that having cables around when using a vibrator isn't exactly exciting or practical. On top of that, modern smartphones are abandoning the headphone jack...

App control

I want to devote a whole section to the final way of controlling the vibrator. Nowadays smartphones are ubiquitous and most of the sex toy companies have jumped into the app bandwagon. Almost all of them have at least one smart or interactive device which can be used with a smartphone app.

Such an app could be used to enjoy solo play (for example synchronizing the device to music, like in the previous section), to maintain intimate contact in long distance relationships or enable discreet public play.

In some of the previous cases, internet connection is a must, in others not really, but the manufacturers force it onto their consumers. I'm personally very privy to personal data and hate sharing any info which is not necessary. I therefore think that projects like Buttplug.io are great initiatives to take control back and create our own experiencies in the field of sex toys.

But, coming back to the smartifying a dumb vibrator project, integrating my device with buttplug.io and building a client based on that library would be too much work.

Therefore I decided to build a webapp which would be hosted in the microcontroller which controlled the vibrator directly. I switched the Arduino UNO for a Wemos Mini D1 based on the ESP8266 and started writing some code based on my previous Cyberspikes project which used a similar control interface.

I designed four vibration patterns, rising intensity, sinking, sinusoidal wave and a random surprise pattern. I also added a way to control the maximum vibration power in order to avoid an uncomfortable situation.

I wanted to represent all of these patterns using mathematical functions instead of lookup tables, in the first place to avoid using more memory than needed (pretty constrained in a microcontroller) and to be able to generate a wave with the highest resolution possible.

One programmed, I tested the app by plotting the output voltage values, instead of connecting it to the vibrator directly. In the following video you may see the test:

After ironing out a few bugs and seeing everything was in order I assembled the prototype and tried it out with a spoon and come sugar. I really didn't know how to convey the vibrations in a safe for work video without relying entirely on sound:

Future work

It's clear from the previous video that the vibrator cannot really be used with a few cables and a protoboard hanging around. Having finalized the software part the next one would be minification of the system.

The cool thing to do would be to design my own PCB board and create a self-contained small device, but I'm not knowledgeable in that field yet. Therefore I would chose a smaller ESP8266 development board, squeeze all the electronics together and design a case I would later on 3D print.

Exploring the control ways I didn't get working would also be a nice way to improve this device. I'm still interested about controlling the device with a thermometer, I haven't seen yet a sex toy using temperature this way.

Finally I still have to test what would happen if I exceed the 1,5 volts my vibrator is designed for, I think it could easily take at least 3V. It's just a DC motor and the vibrations could be much more powerful (= more fun?). But on the other side I still want to respect the device's limits, I wouldn't like to use an unsafe device.

Some time ago reading some hacking sites on the internet I saw an article talking about how awesome it is to type on a mechanical keyboard, since then I saw mech keyboards mentioned everywhere.

My first idea was to go the easy route and get a prebuild mechanical keyboard, I had no idea which switches to choose, what a customizable mech keyboard could do or the great world of custom-build mech keyboards.

Choosing the size and distribution and other extras

I soon learned some aspects critical to a mechanical keyboard are its size and distribution.

Size

This one was quite a revelation! Not every keyboard has hundred-and-something keys and a Numpad (I love Logitech's K120), many mech keyboards I saw were on the smaller size.

Here's a almost-comprehensive outline form onsitego:

The layout I preferred were the 75% or 65% ones. Losing the arrow keys was a no-go for me and he function keys are a nice extra for IDE shortcuts.

Key distribution

One could say there are only two distributions: ANSI and ISO. There are surely some others, but these two seem to dominate the market.

The ANSI distribution is the American one, and the ISO is used mostly in countries located in the EU. Each country having its own ISO variation (ISO-ES, ISO-FR, ISO-IT, ...)

Here's an image from wikimedia displaying both layouts:

Even though I'm based on an EU country I wanted to get an ANSI keyboard, as I had heard from some programmers in my country that this simplified programming. That's because generally programming languages use symbols that are more easily accessible in an ANSI distribution. In most ISO distributions one has to use key combinations to get those symbols.

Additional extras

Mech keyboards normally have some extras only seen in premium keyboards (may they be mechanical or membrane ones), like key backlightning, RGB LEDs to show off, or some extra function keys. Some keyboards also allowed firmware modification using QMK.

I didn't need any of them, although key bakcklightning would have been a nice extra and custom firmware an incredible addition.

Finding the perfect keyboard

So, with a layout and distribution in mind I started looking for keyboard shops. Finding an online shop based on Europe (to avoid import taxes and tolls) was quite an adventure.

But in the end I located https://mykeyboard.eu/ and https://candykeys.com/. The thing is, mech keyboards seem to be such a wanted item that those shops hardly had (and have) on stock the keyboards I laid my eyes upon.

And which ones where those keyboards? Well, my favourite one was the Varmilo VA87M Panda, a beautiful TKL keyboard.

The Vortex RACE 3 was also a very strong contender, a compact (although slighlty boring-looking) 75% keeb.

But in the end I didn't get any of the ones I had lined up. What happened?

Keyboard kits and DIY

Looking through candykeys catalogue for the Nth time I stumbled upon their DIY keyboard kits section. And OMG, that was exactly the way I like to roll! even though my maker-fu has a long way to go I dig building my own devices, and having a sort of path laid upon to help me build a fully custom mech keyboard was a great opportunity.

Building a custom keyboard consists on getting all components that make up a keyboard and assembling and soldering them together. Those components are:

• A case to put the keyboard into an protect it's most delicate components
• A PCB where the key connections are printed and connect to the CPU (the keyboard's brains)
• A metal plate to add robustness to the build
• Many, many switches, the electro-mechanical components which convert the finger taps into electrical signals which get converted to symbols on the computer
• Keycaps, the plastic bits that are mounted on the switches and are tapped
• Stabilizers some metal wires and plastic bits that help anchoring the bigger keycaps so that they don't fly away when pressed

I quickly got a crush on KBDFANS' KBD67 MKII keyboard, but all of it's main components were not on stock and my patience was already pretty depleted.

And this way I found the Katana 60 v2 keyboard.

Katana 60 v2

The Katana 60 v2 was a bit of an imposing keyboard which seemed difficult to get used to, given it's small size and unique key distribution. It's based on the SemiErgo Layout, based on a 60% size keyboard, reducing stress on the wrists.

RominRonin was the brave one designing a PCB to translate the SemiErgo concept to the real world, resulting in this beautiful keyboard. Even though I slightly feared such a drastic change (from a 100% ISO keyboard to this), I loved the idea of having such a distinctive keyboard.

I ordered the PCB and plate through candykeys, switches (gateron browns) and stabilizes through mykeyboard.eu and the cheapest case and keycaps i found on Amazon.

LED mod

While waiting for everything to arrive I fantasized with the idea of adding a RGB LED to the keyboard as a status indicator for caps lock and other functions. And I found this mini-guide from a Reddit user.

But there was a problem, this user added the LED to the Katana 60 V1 board, and I had the V2 with some slight modifications. Thankfully the Katana 60 being an open hardware keyboard I downloaded the Eagle files from its repo and found the PCB traces I could solder some wires to to get the LED working.

As soon as I received the PCB I added the LED:

I added a semi-translucent band aid on it to diffuse its light and make give it a smoother appearance.

Assembling the keyboard

Once all pieces arrived home I started to assemble the keyboard. The Katana60 allows the keys on the lower row to be distributed in different ways: a single spacebar or two split spacebar ways. I decided to set a single spacebar, but the plate didn't work with this layout so I slightly filed some edges. Aluminium being an easy metal to work aided this modification.

The order of assembly is critical in a custom mech keyboard. First the stabilizers are added to the PCB. There are some tricks to reduce their noise which can only be applied at this stage!

After the stabilizers are set, the plate with some switches to align it are connected to the PCB. With the plate on its place one places the rest of the switches, they are anchored mechanically to the PCB by way of simple pressure.

Then the switches are soldered to the backside of the PCB to form an electrical connection between them. Afterwards the whole assembly is placed on the case and screwed together. Finally I placed the keycaps.

But... whoops, due to the katana having quite an uncommon distribution and my decision to use a 2 unit backspace, when the board wasn't designed for it meaned I didn't have the keycaps for some of my keys.

I found this magnificent openscad project by rsheldiii KeyV2. It's a parametric key generator! I specified the sizes of the keys I needed in the program and rendered a model which I printed in 3D. And done, I got the keys I needed to complete my keyboard.

The firmware

But wait! We're not done yet. The keyboard on it's own is a dumb device without function. It needs a firmware in order to communicate with a PC and relay the keys that are pressed.

The Katana 60 can be flashed with the QMK firmware. It's a super-configurable sort of operating system for keyboards. I designed my layout on an online editor, exported it to my PC, added some modifications to make the LED work and compiled it.

After flashing the keyboard with it I had my own Katana 60 V2!

Some final thoughts

After a year of using the Katana 60 I'm still delighted by it. I love the feeling of mechanical switches and the keyboard's layout, although I'm not using it correctly.

I don't think I'm missing out on having more keys, the Katana's distribution and my custom take on it are more than enough to cover all my use cases.

Additionally, having a small keyboard allowed me to place the mouse further to the centre of my desk, reducing the travel distance from keyboard to mouse.

Thanks Rominronin for such an incredible design!

Being always open for a challenge I agreed to build a replica for them, but improving upon adafruit's version. I wanted to add wireless capabilities to change colour and pattern, easy to use snap-connectors for the spike strips, the possibility to add one or two strips and a way to use a universal USB power supply (probably a power bank).

3D Printing the spikes

I got some Smartmaterials Flex filament to print the spikes which worked pretty well. The only issue at this stage was fitting the spikes inside my 3D printer bed, which isn't really large. So I split the model in two smaller strips and the fused them together using a soldering iron. The result was better than expected.

Afterwards I simply glued them on top of a waterproof neopixel LED strip and put some magnets on the under-side so that they stay in place when worn.

I also 3D printed a simple PLA case to house the microcontroller which would drive the light strips. I added an abstract design to the housing to make it look more cyber.

Electronics

I decided to go for the smallest ESP8266 based development board in order to reduce the controllers footprint and be able to add wireless remote control capabilities to the spikes.

Following this criteria I chose a Wemos D1 mini board. I would only need to use two digital pins for sending the neopixel control commands, and that board even had a 5V port which was really nice to get the power from for the LED strips. Using 3,3V (the only voltage available in many ESP8266 dev boards) would have impacted in the LED's brightness.

Initially I used Adafruit's neopixel library to control the strips, but I found some difficulties which limited what I could do with the cyber-spikes. Looking around I stumbled upon a great LED controller library named FastLED which is incredibly powerful. After studying some of their example programs I picked the patterns I liked most and whipped up a simple program to light up the stripes.

Snap connectors

One of the other improvement I metioned are the snap connectors. I bought a pack of 3 wire male and female connectors and soldered them to the LED strips and the microcontroller which would control them.

The microcontroller would send the secondary light strip the same signal as the primary one, so connecting a single stripe on any of both connectors wouldn't really matter. This way the user could wear a single or a double spike strip.

Remote control

What fun are RGB LEDs you cannot light up as desired? The cyperspikes needed a remote control! But, having to carry an IR blaster with you everywhere isn't the most user-friendly way to do this.

Guess what's in (almost) everybody's pocket: A smartphone. The thing is, I didn't want to make a native app to control the lights, so I decided to make a web app and serve it from the ESP8266 directly.

When booted up the microcontroller sets up a WiFi access point. The user can then connect (temporarily) their phone to the microcontroller and access a webpage its serving, which contains a few buttons to configure the LEDs. Each button sends a POST message to the ESP8266 which changes the light pattern accordingly.

The UTSUND string is battery powered. It stays on for a few ours and automatically powers off. Being a bit of a couch potato I thought of making some sort of remote control for the lights. On the one hand some sort of receiver would be needed on the lights side, and a remote on the other one. Trying to avoid reinventing the wheel and using what`s already available I thought of using my smartphone to control the lights.

To control the lights I needed a microcontroller and a relay, but with these lights being such an light-powered device a MOSFET would be enough. As wireless-capable microcontroller I chose a NodeMCU based on the ESP8266. The NodeMCU would be connected to my local home WiFi network.

I didn't want to set up any cloud servers for this task, as I wanted a simple solution and had no need to turn the lights on or off when outside my home. Taking this into account and trying to avoid developing a specific app for the smartphone I thought the NodeMCU could act as a server showing a simple webpage to act on the string light. This would also be a universal solution, I could use not only my smartphone but any device to switch the lights on or off without any setup.

So, taking all of this into account I set up a simple test with a single LED on a breadboard.

Then I connected the same setup to the string light, using the NodeMCU to power the lights directly. The original power source is 3 AA batteries, that would be 4.5V aprox. The NodeMCU operates at 3.3V, but the USB port receives 5V. In order to take advantage of these 5V I soldered a wire to a component skipping the power circuit in the NodeMCU.

After checking that everything worked OK I made the final connections between the NodeMCU and string light and emptied the batteries case. I reused it to house the microcontroller, MOSFET and wires.

I reused the original Utsund battery case in order to hide the electronics within.

Lately I was using some resource-intensive programs at work and couldn’t reserve screen space for the task manager, in order to keep tabs on the CPU and RAM use.

To solve this problem I decided to create an external device which could show those values I was interested in. But I also wanted to create something striking to have on my desk. I decided to use a pair of analog VU-meters to reach the desired goals.

PROGRAMMING

The system is composed of a server and a client application.

Server

The server app is a .NET console app I quickly whipped out on Visual Studio. I didn’t want to invest any time here.

The program reads the system resource values, adapts them to a 0-100 percent range and sends them to the client arduino app via serial communication.

Client (Arduino)

The client app is an Arduino program. It reads the values passed from the server via serial, does a little processing and shows them in the VU-meters.

As you’ll see in the Electronics section the VU-meters aren’t but a pair of modified voltage meters. In order to show any value on them the arduino just needs to generate a voltage in a given range. That’s accomplished via PWM output.

There are also a pair of RGB LED strips. These change colour based on the load of the system. Up to 60% load the Arduino just uses a blue-ish tint. But when the values climb above that threshold the colour becomes warmer, ranging from yellow to red.

To do this colour progression I decided to use the HSV colourspace and alter the hue proportionally to the value shown in the VU-meter. This was an incredible simple way to implement this functionality.

ELECTRONICS

The base of this system is an Arduino UNO board. Only 3 of the PWM outputs are used to make everything work.

VU-meters

The VU-meters are a pair of voltage meters I bought on ebay. Their original range was 0-15V, I opened them up and changed the resistor they had inside to make them work on a 0-3.3V range, originally I was going to use a NodeMCU board as the brains.

I then scanned the voltmeter’s panel and designed with GIMP a new one for CPU values and another one for RAM values. I printed them and stuck them over the original panel.

To show a value on the VU-meter the arduino just has to generate a specific voltage on the 0-3.3V range via one of the PWM ports.

LED sticks

I used a pair of LED sticks to give a sort of backlight to the VU-meters.

Originally I didn’t want to create backlightning, just a way to warn me when the meters reached their max values. To achieve this I wanted to use a simple red LED, but I didn’t have anywhere to put it and didn’t have the tools to modify the VU-meters’ enclosure.

I then used the LED sticks to make the meters glow when their max values where reached, but wanted to create a cooler effect. All in all, RGB LEDs are capable of much more than just what I intended to use them for.

Due to some problems with Adaruit’s neopixel library I found and ended up using the fastled library to drive the LED sticks. This library also allowed me to use the HSV colourspace and solve the color-progression problem easily.

EXTERIOR

In order to make the device self-contained I started to design a simple enclosure with CAD software, but soon it become clear that it would be pretty big. In order to avoid printing large pieces I only printed the front panel, which on its backside has supports for the heavy VU-meters.

The rest of the enclosure was made with a cardboard box and taped together.

PUTTING IT TOGETHER

A designer will draw a quick sketch of the desired GUI, and then photograph it with the Imek app. Imek will then analyze the sketch, extracting the sketched GUI components and the way they are ordered. The app will then instantiated native OS components and create a GUI in the same order as in the sketch. The end result will be a mock GUI created the same way as in the sketch.

This project will contain:

• Image processing –> To parse the sketch
• Machine learning –> To interpret the parsed figures
• Android programming –> To dynamically generate a GUI and present it to the user

Part 2 - Element Extraction

In the first iteration of the element extraction code I managed to read a squares array and create subimages with each individual square.

This is an example of the input image:

The image was manipulated using the following methods:

1. Conversion to grayscale image
2. Gaussian blur applied, with a 51x51 kernel, in order to eliminate small imperfections on the image
3. An adaptive Gaussian thresholding is then run on the image in order to create a binary image containing the most prominent lines
4. Smoothing of the resulting lines from the previous step a. 4 iterations of dilation with a 7x7 kernel b. 2 steps of erosion with a 7x7 kernel c. A Median blur is applied in order to eliminate small pixel islands which may have appeared after the dilation step

After the image is preprocessed opencv’s findCountours function is called with the cv2.RETR_EXTERNAL and cv2.CHAIN_APPROX_SIMPLE parameters. This will find the external contours of the squares.

The resulting contours are then filtered, in order to eliminate the smallest ones, which may be small pixel islands which are not a square. This step uses a hardcoded minimal area, but it should use a relative area measurement based on the image size.

With the final contours on our hands, the code proceeds to created bounding rectangles on the contours, which will contain the squares we were looking for. The following image shows the input image through some parts of the process:

With the bounding rectangles on our hands the original image is reloaded and reprocessed again using similar steps to the ones previously described. The resulting image is used for the extraction of subimages based on the defined bounding rectangles. This process results in a list of subimages containing only the squares:

Part 3 - Element Extraction (Continued)

The third iteration of the image extraction script is a refinement on the first one. The new script lets the user play with the parameters used in the algorithms freely and save the results when the expected result is met.

This version does also take into account the order in which the elements are ordered vertically.

I used the script to extract 3 types of elements to train the future classifier:

Part 4 - Classifier

To know what Imek is about visit the introduction blog post.

The next step for the Imek project is to be able to classify the individual elements detected on an image. Knowing which element is represented with each image will make it possible to build the UI in the next milestone of this project.

I decided to use tensorflow as the machine learning framework to use. It struck me how difficult it seemed to simply get going and create a simple classifier. Well, after some wasted hours I stumbled upon this blog post in which the author used tensorflow’s example scripts to train a model and classify an image. It seems that using the examples provided by the tensorflow authors is a much better place to start than documentation.

With the retrain.py script I was able to quickly retrain the mobilenet_1.0_224 network. With label_image.py I tested it quickly, obtaining satisfactory results.

I then continued by taking tensorflow’s Android example and extracting the ImageCalssifier. I then built a simple prototype app which did the same classification I did manually on Android.

Part 5 - App demo

To know what Imek is about visit the introduction blog post.

Here’s a demo of the prototype app I’ve built over the last days. It takes a picture of a sketched interface, applies the image processing algorithms described in the previous blog posts and labels the extracted subimages with the retrained mobilenet neural network.

Having classified every UI fragment the app rebuilds it with native android components. For now it only builds interfaces with ImageViews, TextViews and RadioGroups.

The code is available in the Imek repo.

Kikimora was born form an idea given to me by colleague at work. The idea was to create a simple device, which with the aid of two LEDs would indicate if a computer system is up or down. The device could be set on top of any computer screen in order to make it easy to check out.

I specified further what I would try to accomplish and thought of the following: The device would be configured to ping or somehow GET a webpage every X seconds. If the page responded with a 200 HTTP code, let’s light the green LED, any other message: red LED.

After that I added some requirements to the device: to make it self-contained, as simple as possible to use, and small (in that order). So, I started the hunt for an arduino-like platform with embedded WiFi capabilities, and landed on the ESP8266 module family. I bought an ESP8266-01 and ESP8266-12E, because they are dirty cheap. I ended up using the 01, the reason is simple, I liked the restrictions imposed upon me by using such a “simple” device. It only offered me a WiFi link and 2 GPIO ports (though I’m sure someone has already invented some kind of hack to use the RX/TX ports for GPIO).

PROGRAMMING

After a long afternoon programming the ESP8266 I managed to cram a program which did the following:

• Setup a Wifi connection to a hardcoded network
• Setup an internal timer (based on the OSs functionality, and not some cheesy software hack)
• Host a configuration page to setup the page to monitor
• Write some code to light a LED or the other based on the target website’s status

I took into account the hardware limitations and avoided to work too long in the device’s main (and only?) thread.

The code is available in the project’s repo, linked at the start of this article.

ELECTRONICS

The electronics were pretty basic, in the end it’s just a microcontroller and two LEDs. In order to pursue the “simplicity” aspect of the project I wanted to supply the juice via an USB port, like most handheld devices around us. Two problems arise: 1. A micro-usb port is probably fragile + difficult to solder in a perfboard. 2. USB provides 5V… and the ESP8266 needs 3.3V.

The first one I solved using and old-school USB-A female port. To solve the second one I wanted to get away with a voltage divider, but due to the laws of electronics it wound up to be a difficult task, so I summoned all my patience, order a bunch of voltage regulators on ebay and put the project on hold for a week. When they arrived everything worked perfectly.

The schematic for the circuit is as follows:

First, sorry for putting an ESP8266 which is the wrong module in the schematic, but I didn’t find the 01… Secondly, there is no much mystery to it. The resistor after the LEDs should avoid the build up of “infinite” current, and that’s all.

The first prototypes I tried out in the proverbial protoboard. The definitive circuit was build on a perfboard. Pretty badly. I should get a better soldering iron and something to hold the board and components while I solder them. Sorry for the terrible craftsmanship you’ll see in the photos.

There are two more things I would like to talk about: First the operating voltage. As you have probably notices there is a voltage regulator which reduces the USB’s 5V to 3.3V for the ESP8266. The bad thing about this is that the LEDs will be slightly underpowered, emitting only a very faint light.

Secondly, the ESP8266 is mounted on some pins and not directly soldered to the perfboard. This is so in order to make it easier to reprogram the microchip in the future without much hassle. The downside: The high occupied by the whole ensemble, which is quite a lot for a device of this size.

EXTERIOR

The exterior is a simple parallelepiped or a rectangle with volume. I made it so that I could be easily opened in the middle and accommodate all the components (mainly regarding height).

The bottom part of the case takes the board. The perfboard is mounted on two screws and the edge of the case, leaving a bit of space beneath for some cables and solder blobs. The top was build with the height constraints in mind. There are two holes for the LEDs to go through, and a notch for the upper half of the USB port.

I designed the case in FreeCAD (in a pretty unorthodox way, regarding good practices). The top half was a bit of trial and error thing, it was difficult to define the right height for the piece and LED hole diameter, taking into account my printer’s tolerances and accuracy error.

I exported the models to STL meshes and converted them to gcode using Cura 15. Then I printed the case in my Prusa i3-model 3D-printer in PLA plastic.

PUTTING IT TOGETHER

To finish it of I glued the LEDs to the perfboard with some silicone sealant and decided to bend the voltage regulator beneath the ESP8266 board in order to avoid it hitting the top of the case. As it doesn’t heat up too much I think it won’t cause a problem there.

You've seen the complete build on the beginning of this article:

The ESP8266’s code, CAD and STL models are all open source. You can find a link to the repo.

Bannik will be formed by a hardware device and a software counterpart to control it. The hardware will be acquired keeping a low cost in mind and using what I have laying around. Regarding software, only open source systems will be used.

I visualize an unconventional “semaphore” regarding current standards. The idea is to use a single multicolor disc to show a status quickly (via colors) and a screen to get further information on an alert. Some sort of acoustic signal could be an improvement. The device could be mounted on a wall, or placed on a desk.

Regarding the screen I would like to use some sort of panel that is easy to read, I have my mind set on an e-paper one. Going back to the “disc”, something regarding RGB LEDs could be useful.

I’ve already specified the innards of the bannik monitor:

• A RaspberryPi Model B (1st gen), it will run on Raspbian Jessie Lite
• A TPlink TL-WN722N, to bless the RPi with WiFI connectivity
• An Amazon kindle 4, as readable “more info” screen
• Two RGB LED Rings, 24 and 12 LEDs respectively
• To power all of this I got a 25W (5V, 5A) power supply

Let’s talk about the list: First, the RPi. I had it laying around. I think current RPis are a bit overpriced, but very capable devices. I considered an Arduino, which would be a better candidate to interface with the hardware, but, it’s easier to make an RPi host a REST API + other future improvement ideas, than an Arduino. Regarding the WiFi dongle, I’ve got nothing to say. It was cheap on amazon, some people said it worked, others that not. Well, when I got it I plugged it in the RPi and worked flawlessly. No driver installation, no kernel module recompilation, no nothing.

The kindle is my answer to an easily readable screen to get further information on bannik’s alerts. But, multiple problems arise: How am I going to show content which isn’t a book? How am I going to power it? And… The screen is too small. At least I bought it pretty cheap from an auction on ebay, let’s see what I can do with it.

LED Rings… They’re a copy of Adafruit’s wonderful NEOPixel Rings. I would have bought the Adafruit’s ones, would they be easily available in my country. But it’s easier to search ebay for cheap electronic components than to visit every European Adafruit distributor and find in their online stores what I’m looking for. Slight problem… 36 Neopixel LEDs require 60mAx36=2.16A. No way I’m pulling that from the RPi.

So here’s the solution to my power-related problems: A power supply. Putting an internal power supply in the bannik device will cover all my power needs an minimize the cords running to it. A single cable from the outlet to the device is all I need. I worry 5A may not be enough to power all the devices, it’ll probably be a close one.

And now… a shell to hide all of this. I will use balsa wood or plywood to make a case. 3D printed supports will hold everything in place inside. The problem lies on the front of the device. Cutting out a circle in a wood sheet to show the LED rings won’t be a problem, but cutting out a rectangle for the kindle may be, so I’m considering 3D printing the whole front of the device.

Planning how to place everything inside, without the components interfering with each other is proving difficult, especially trying to minimize the footprint of the device.

A low-cost intelligent ECG. An android app that analyzes the health of the patient’s heart using an Olimex ECG Shield connected to an Arduino board. The data from the shield is relayed in real time via Bluetooth to an Android-running tablet, there the data is shown to the user. A series of complex algorithms are run to analyze the data, followed by an expert system which diagnoses the patient. All of this for about 70 euro.

Here's a demo video of the device being assembled and used to perform an ECG:

Super-duper mega Pong is a pretty plain clone of the game Pong. Some wonky “physics” are involved to make the game unpredictable. It’s a beginners project using the libGDX library.

A simple sonar made with Arduino. An ultrasound module attached to a servo sweeps an area and plots the distances to nearby objects in an Android-powered tablet. The data is transmitted via Bluetooth and it makes use of a custom View object to draw the data on the tablet’s screen.