Adruino updates: APC w/ESP8266 and OLED LCD

[Print This] By mjhammel ~ November 20th, 2015. Filed under: Arduino.

So my little aeroponics project is growing beyond its real needs, but once you start assembling legos it’s hard to stop.  I’ve added an ESP8266 for wireless connectivity and a really small OLED LCD display.  The latter will let me monitor the system without either serial (FTDI Basic) or wireless connected.


The breadboard really isn’t necessary at this point. The only thing it holds are the LED and its resistor plus Vcc and GND busses.

Fritzing made it easy to layout the prototype on a breadboard, though after working through cleanup of the schematic and the PCB I found some problems.  First, I intended to use a USB breakout board from SparkFun connected to some headers on the PCB.  But if you try to wire this in Fritzing the circuits are not closed at the headers and Fritzing gets confused.  Instead, I found a USB part (from SparkFun) in the Fritzing parts libraries that had through holes.   Through holes are important because I need to place some traces on the PCB on the bottom side of the board.  The USB  part had connectors that were labeled Vcc and Ground which allowed Fritzing to close circuits and allowed me to create traces the way I needed them.


The layout is about 8″ x 5″ which could be a bit pricey, not to mention difficult to fit in a decent sized enclosure. It will have to be scaled down.

The next problem came after the PCB was fully routed.  The size of the PCB makes it prohibitive to get made.  OSH Park charges about $5 per square inch.  With my PCB at 8.8 x 5.3 inches that cost for a small production run (3 boards) would have been around $200.  Not horrible but really, this is such a simple project why waste the money?

So now the plan is this:

  1. Remove the FTDI Basic. The Pro Mini just needs headers to connect the FTDI Basic.
  2. The OLED LCD doesn’t need to be on the PCB. It can be connected with wires. This will allow the display to be freely placed in the most appropriate location without respect to the positioning of the PCB.
  3. With these changes I should be able to mount the ESP8266 on the back of the PCB and
  4. move the TinyRTC and USB power to a second PCB, mounted vertically with spacers. That will reduce the costs of the PCBs significantly. In fact, I should be able to do the whole thing with perfboard.

So my cost goes next to nothing.  In the end I should have two perfboards mounted together with spacers with the LCD mounted on a third perfboard (or without possibly) on some user-facing wall of the enclosure.

I still haven’t gotten to the enclosure but that will come after I find out how big this thing will be.  I still have to add in the 4 channel relay board too, plus the wires connecting all the boards and the power outlets together.


Just posted this, then went back to review the layout in Fritzing.  I realized that the ruler part I used to measure the PCB had two sides: inches and cm.  Guess what idiot was reading the cm side?  Yeah.  That’s the guy.  So it looks like the whole design may fit completely on the perfboards.  The PCB would only be 3.3″ x 2″.  Much more affordable.

Just goes to show:  there is always one more thing to review….

Related posts

Embedded project: 4 channel Arduino-based power controller

[Print This] By mjhammel ~ November 5th, 2015. Filed under: Arduino, Hardware, Holiday lighting, Raspberry Pi.

The Sainsmart 4 channel relay board. Digital inputs (lower right) control standard 110v through the four relays (large blue boxes).

One of the directions my PiBox project is headed is as a central hub for sensor management.  This implies the ability to communicate between the PiBox and the sensor node using BLE or Wifi.  PiBox can do that already over Wifi (and supports basic BLE communication though lacks a meaningful UI or automated registration) so I only need to start work on sensor nodes.

Now it’s hard for me to toss together a temperature sensor node without having a purpose for knowing the temperature.  So whatever node I created needed to have a purpose.  Enter:  my newly configured aeroponics systems.  This system uses a 5 gallon bucket with a water pump inside to spray nutrients on the roots of the plants.  Lamps are hung from a support structure (a wire frame shelving unit) .  Lamps and pump are plugged into a simple manually configured timer.  Nothing special there, except for a really leggy tomato plant that has yet to fruit.  But that’s a different project.

So I thought to myself:  what if I created a programmable timer for the lamps and pump?  I could use an Arduino for the brains, plus added components to make it clever.  That’s my sensors purpose:  handle power on and off of the lamps and pump for the aeroponics systems.  So was born apc:  the Aeroponics Power Controller.



The aeroponics tub and manual timers (lower left)

I’m sure the first comment will be:  but that’s not a sensor.  Yeah, it’s not.  But the concepts are the same:  embedded device controls some physical components based on input from a remote management interface.  For apc the physical components are power outlets.  APC’s initial implementation will use four controlled outlets, each capable of being programmed independently.  It needs to be able to run scheduled events (power enabled and disabled on each outlet).  There should be some way of knowing the status of the controller:  running or not.  And there should be some status indication of what’s happened, re: a display of some kind.


Based on the design I need the following components.


Pro Mini (center), RTC (lower left) and USB breakout plus LED. Only thing not shown here is the FTDI USB board, which connects to the pins on the left end of the Pro Mini.

These are the main off-the-shelf boards I need.  I also need a breadboard to hook them all up, an LED and associated resistor and a bunch of wires.  I discovered, after finishing the version of the project, that soldering is limited to adding headers to the boards and a few wires to the headers.

Why the Mini Pro?  Well, for one thing there were on sale at SparkFun back in March 2014 when I bought a bunch of similar boards without having a specific purpose for them.  So it’s been sitting around for awhile waiting for an inspiratio on my part.  But it also has a very small footprint without all the bells and whistles (re: bulky connectors) required for other beginner projects.  And it has plenty of computing horsepower for this kind of project.  It’s actually way overkill.  But I already had the board, so what the heck.

The TinyRTC is a real-time clock with a battery.  The battery isn’t really necessary but it’s nice to have if you lose power.  Though without power the apc won’t work.  Dealing with that problem is beyond the scope of this project.

The 4 channel relay is the board that converts the control signals from the Arduino into the enable/disable actions of the power outlets.  The relays are optically isolated so when each is enabled they make a loud click sound.  You know they’re working if the click is loud and distinct.  If it’s a quiet click you probably don’t have enough voltage on the control lines of the 4 channel relay.

The FTDI Basic breakout board is what you use to connect your Linux box to the Mini Pro.  It’s what you use to upload your programs to the Arduino.

The USB power from your computer is enough to handle programming the Arduino and running the program.  But it’s not enough to change the state of the relays.  So a USB breakout board is used to connect a wall-wart power supply with 5V output to the APC.  This powers the Arduino and the relay board.  Note that you can connect this power source AND the power from the FTDI Basic at the same time.  It won’t hurt the board (as far as I can tell).  That’s how you test your code.

Hook Up Guide

The nice thing about this project is how it can be done with a series of off-the-shelf breakout boards.  The Mini Pro has plenty of digital lines to connect to more than one board.  Strangely the pins I need from it for SDA/SCL to talk to the TinyRTC are not aligned to the breadboard.  That’s a trade-off when using a smaller microcontroller board like this, but it’s a minor inconvenience.  I just soldered two pins to the A4 and A5 holes in the Mini Pro, then used female connectors to wire them to the breadboard at the SDA/SCL lines on the RTC.

Fritzing diagram for first APC implementation.

Fritzing diagram for first APC implementation.

The 3.3V power from the Mini Pro is not strong enough to switch the relays (not shown in the Fritzing diagram) so a USB breakout is used to bring in power from a wall wart that provides 5V input.  In fact, that’s why I added two additional outlets to the original set of four outlets.  The two new outlets are not controlled and I can plug the wall wart into the USB breakout board.  Easy-peasy.  This is why IoT is so popular.   It’s a bit like Lego building, but with wires and pins.


All the code for this project is available from the git repository at  The code utilizes two libraries for the RTC, the Wire library for serial communications and the Timer library for running scheduled events.  The Timer library provides for events to be run on short intervals like 100ms.  The RTC libraries allow checking the current date/time against the schedules to see if an event should be triggered.  Events are simply when the outlets are enabled and disabled.

As a side note to programming:  I’m not using the Arduino IDE.  I think IDE’s are a crutch that prevents you from learning WHY things work.  So I’m using Tim Marston’s arduino-mk to edit and build from the command line.   I added support for opening  serial monitor to it, which you can find in the source code under the docs directory.  I also use screen to create a bunch of virtual terminals in a single window.  This I do because I can then work from different systems (desktop or laptop in different rooms of the house, for example) without losing history or resetting my environment.  Lastly, I use cdtools with my project because I have a zillion projects and cdtools functions make it easy to bounce around them.  Config files for both screen and cdtools are in the docs directory.

Automated Registration and Web UI

The next phase of this project will add networking to the system.  An ESP8266 will be added to provide wifi connectivity.  The ESP8266 uses an AT command set to connect and send and receive data from remote servers.  The plan is to have the Pro Mini perform an initial registration with a remote server running on PiBox.  The registration is initiated with a multicast message from PiBox asking for new devices.  The device receives the message and sends a whoami packet with it’s IP address and a registration token (something akin to a secure key, I think).  Once registered the device can access the PiBox web server via a RESTful interface.  The interface will provide scheduling data for the device and the device can report status back.

Of course this requires a web interface for setting schedules for all registered devices.  Fortunately PiBox already has a web UI with a RESTful API.  I just need to extend it to support IoT devices.  Then I need to create a UI, potentially using a self-defining UI format so devices can create their own configuration options without having to place the knowledge in the UI.   If done right this format won’t take up much space on the device because there will be little to configure in each IoT.  That’s the point:  IoT implies a device with a simple purpose that is remotely managed.

Beyond the ordinary

The ESP8266 will connect to the serial lines of the Pro Mini so that means the serial console won’t be available.  There is a neat little trick around that:  a color LCD display.  These devices are cheap, around $10-$20.  I’ve selected a four wire device that uses SDA/SCL lines for data plus a ground and Vcc.  Luckily the TinyRTC board has extra SDA/SCL pins to piggy-back another device like the LCD.  And Adafruit has libraries for these devices that make it very easy to use.

After all this is done there is only one thing left to do: extend scheduling for my holiday lights.  After all, wasn’t that really where this was headed?  I mean, do I really need brains in my outlet timer for a tomato plant?  I guess that depends on if that thing ever starts to fruit….


Related posts

PiBox: bluetooth for video means no omxplayer

[Print This] By mjhammel ~ July 27th, 2015. Filed under: Audio, Hardware, Linux, PiBox, Raspberry Pi, Software Development, Video.

One of the longterm goals for PiBox is to integrate it into the home.  I want to see the PiBox Server provide centralized communication for an array of IoT devices.  One way to do this is with Bluetooth Low-Energy, aka BLE, aka Bluetooth 4.0.  This specification allows very low power sensors to report information to services that can do something useful with the information:  close the shades when the temperature goes up, let the roomba loose when the dog’s sheddings are accumulating, turn off the stove when granny forgets to take her meds.

Before I can do this I needed to get bluetooth classic working on the server.  This turned out to be fairly easy.  The kernel supports it and there is the BlueZ userspace tools, plus the bluez-tools wrappers to make it even easier.  After much procrastination I finally got around to testing these on a server with a 4.0-compatible USB dongle.  After a little fiddling with the /etc/bluetooth/audio.conf and /etc/asound.conf files I was able to play a simple wav file to a bluetooth speaker.  Easy-peazy.

The next logical step was to get omxplayer, the hardware-accelerated video player for the Raspberry Pi that I use to play videos full screen, to switch from the analog output to ALSA.  Turns out this can’t be done, or at least it isn’t done yet.  omxplayer doesn’t support ALSA.  So you can’t play the audio stream from a video over bluetooth.  So much for my drive-in movie with wireless speakers.  I looked at what might be required for this (there were hints from the omxplayer developers) but it seems a bit over my head and would distract from the larger scale goals for the project.  I’m a one man team.  There’s only so much time in a year.

So I’ve started banging my head for alternatives.  The obvious one is to plug an analog-to-bluetooth transmitter into the analog output port of the Pi.  Too easy.  Though I have one on order for the short term (hey, we still use it when we go camping).  But that would also require a larger box to hide the transmitter.  And how do I switch between bluetooth and analog output?  I may not always want bluetooth.  Nah, I’m just kidding.  Everyone loves bluetooth.

Another alternative is to switch to a different video player, like mplayer.  That would get me a software switchable option for audio output.  I tried this on a Model B+ but the performance was terrible since the GPU isn’t used.  First, to get a nearly smooth playback the image had to be halved while being decoded on the CPU.  And there was no way to rescale it to the display size at display time.  So this was functionally unusable.

The next alternative (this is computing and there is ALWAYS another alternative) is to switch boards.  I looked at several competitors like BananaPi an Odroid.  The former looks like a possibility.  But then I took another look at the newest version of the Pi, the Raspberry Pi 2 Model B.  It’s a quad-core with each core spec’d at 1GHz, slightly more than the original Pi’s 900MHz.  Not to mention the possibility of overclocking and additional on-board memory.   So mplayer, which does work with ALSA but which requires the CPU to do the decoding, might just work on this board.

So I placed an order for one Pi 2 which has a slightly different Broadcom chip.  I’ve already rebuilt the toolchain for this new chip.  That’s an adventure in itself, but fortunately doesn’t appear to require major changes to the toolchain.  I’m unclear if I have the toolchain optimized.  It doesn’t use the latest Linaro gcc releases so I’m doubting it.  But if my toolchain shows any possibility of running mplayer then I can always upgrade my Crosstool-NG release to get access to the latest Linaro bits.

Related posts

PiBox field trial results

[Print This] By mjhammel ~ July 2nd, 2015. Filed under: General, PiBox.

mediaserver01-largeThis past weekend I took PiBox out in the trailer for a field trial.  The box was mounted under a cabinet with power and HDMI wrapped around to a 7″ HDMI-input monitor.  Power on worked fine and the system came right up.  But not everything worked as expected.  Here is the summary.

SMB playback

The first issue was with shared files over SMB.  I first tried to watch a movie using my LG Volt phone, which runs Android 4.4.2.  I use ES File Explorer on Android to get to the video files.  The phone could access the files and start the playback but the network would drop out after a while.  I then tried it with a Galaxy Tab 2 tablet running Android 4.2.2.  This worked much better and I watched several movies without interruption.  The PiBox Media Player also worked perfectly.  So I think the problem was with the phone’s wifi.  That’s interesting since I often use the phone at the gym when I run on the treadmill to watch Netflix without problem.  Apparently it works better with 3G than it does with wifi.

One thing of interest here:  the tablet notified me at one point of “unusual magnetic activity” in the area.  I got this when I used an astronomy app (setting up to use my telescope – hey, it wasn’t all work on this field trial, we were camping).  I have no idea how it sensed this but wonder, if it was real, if it had any impact on local wifi network performance.


The media server was cabled to a 7″ display in the trailer.  This required an extra power cable.  Then we also had a digital TV box that added additional power and HDMI cables.  Finally I had a HDMI switch to go between the TV receiver and the Media Server.  That amounts to a boat load of cables that didn’t exist before with the old TV/DVD player.  Fortunately, the digital TV and HDMI switch will go away when I add a digital TV dongle to the Media Server.  But I’ll still have a TV cable.  Even so, I can see that there are still too many cables.  I need to power a display and connect to it with fewer exposed cables.  That would leave just the power and digital TV lines exposed, which is much better.  I’m not sure how I’ll solve that problem yet.

Wifi Setup

The wifi setup works well when connecting to the local router.  Using PiBox Media Server as a wireless access point had some minor problems.  First there was a problem with the way I set ccmp vs tkip.  There are two ways to configure the network:  bui-network-config (the GTK+ app run from the launcher) and the web interface.  I fixed the web interface to do the right thing, which is use tkip, in the web backend (piboxd).  But bui-network-config uses its own code for the same thing.

I’ve fixed bui-network-config but the long term solution is to have bui-network-config use the same backend.  The problem is that bui-network-config was designed as a standalone tool that can be used outside of PiBox.  So the question is how to maintain that capability.  I think the solution is going to be to have a shared library that both bui-network-config and piboxd can use for this purpose.

Multiple USB sticks

Something that kind of shocked me was a problem with multiple USB sticks.  With one stick plugged in videos were available.  With two or three they were not.  This may have been a problem with the USB ports.  I still need to check on that.  Another problem may be with how VideoFE handles the databases created by VideoLib on the sticks.  Or it might just be that the sticks were messed up.  This simply needs more research.

Video Browsing

Field trials are good to find out just how usable the device really is.  In this case I found that the simplistic alphabetical list of videos was slow to move through using the keyboard’s arrow keys.  I also found that, for some reason, the poster art was slow to load (or at least appeared that way).  This poster issue is new, as I’d not seen this problem at home.  Again, it may be due to networking issues at this site.

The important issue here is that I need a way to fine tune the list of videos.  Pressing a few keys should be sufficient to reduce the displayed list.  I would need a way to cancel that search (to get the full list back) and/or a way to timeout the current search.  If it does timeout, does the display go back to the full list?  How does the user know what the current search state is (as in timeout state, etc.)?

There’s a bit of work to do here.




Related posts

PiBox prototype cases

[Print This] By mjhammel ~ June 11th, 2015. Filed under: PiBox.

It’s taken more than 2 years but I’ve finally finished the first versions of the PiBox Media Server and Media Player.  The cases are stained wood boxes from a hobby store but they provide me with information on how to deal with layout in a 3D printed case.  It also required me to look at acquiring parts such as specialized cables and connectors.  Some of the small breakout boards I used can be combined into custom boards in future versions.

The wiki has a set of pictures with descriptions and there are more picture in the gallery.

The goal of this project has always been to build a complete software system from scratch using off the shelf hardware as much as possible to create a self contained set of devices that networked with each other.  The hardware is all off the shelf and the few breakout boards are open hardware designs.  The software was built from the ground up starting with a custom cross toolchain up through the custom applications and UI.  It’s even getting some Android apps to go with it (future development).

The only thing that didn’t come out as I’d hoped is the projector used in the Media Player.  I was hoping to find a development board for this but all I can find is a “low-cost” pico projector.  And integrating the TI DLP chip is a little beyond my scope at the moment (maybe with time that will change).

But it’s a working set of prototypes.  I plan on deploying them this summer on our camping trips.  I used them last year without cases.  This past winter our trailer was broken into and the TV/DVD player was stolen.  So now the Media Server, with a 7″ HDMI monitor, will replace it.  The only thing missing from it is digital TV.  I have a dongle but wasn’t able to get omxplayer working with it yet.  That will come with more work this summer.

For now, it’s a start.  If you read this and look over the photos then drop me a line and let me know your thoughts.

Related posts

Apache 2.4 changes

[Print This] By mjhammel ~ June 10th, 2015. Filed under: General.

I’ve been using Apache 2.2 for quite some time.  Lately I’ve noticed CentOS and other distros using Apache 2.4.  The new Apache breaks a bunch of stuff, as does newer php.

One thing that needed changing is a switch in directory listing tools.  I have an archive on my web site that is running Scott Evan’s Indices software.  It’s just a prettifier for file listings but it worked well and was easy to set up and configure for my personal taste.  But this doesn’t work on Apache 2.4 for some reason.  I haven’t figured out why.  However I did find an alternative that works.

h5ai provides more functionality out of the box and is even easier to configure: copy a directory to your document root and add a new path entry to the default DirectoryIndex.  I don’t have an example of this because so far I’m just using it behind a firewall at work.  But it’s quite nice.  Still need to look into customization options.  But out of the box it’s quite useful.

Related posts

cdtools, because who has just one project?

[Print This] By mjhammel ~ April 6th, 2015. Filed under: General, Linux, Raspberry Pi, Software Development.

When I was just getting started with embedded development I found many tutorials on how to perform cross compiles required setting up some shell functions and variables before working on builds.  This is a necessity for embedded work because the embedded build for the target platform won’t use the same toolchain and build libraries as the host platform.  Unfortunately, the setup for the various publicly available build systems were all different.  Angstrom/OpenEmbedded vs Yocto vs Buildroot vs Mentor Graphics vs whoever:   they all have their own setup.

Then I created my own build platform, PiBox.  I did this mostly to teach myself the whole platform bring-up process.  One of the side effects of this (along with having to build custom OPKG packages for PiBox) was the need to bounce from one build to another quickly.  I wanted them all to use the same navigation commands and automatically point where ever they needed to with environment variables.  Thus was born cdtools.

Use case

Let’s say I have a build system for the toolchain, kernel and root file system for my Raspbeery Pi (re: PiBox).  I do out-of-tree builds and packaging to avoid clutter when looking for updates with git status.  So while my source lives under …/raspberrypi/src my builds are under …/raspberrypi/bld and packages generated from the build are under …/raspberrypi/pkg.  So far so good.  That’s not overly complex.

But now I need to bounce over to a kernel tree and make some updates and test them.  The updates I end up with will be integrated back into the PiBox build but before that happens I want to test them in a kernel-only tree (remember: PiBox is a build of multiple components – it is, in fact, a metabuild that downloads components, unpacks them, patches them, builds them and packages them).  So how do I bounce over to the kernel-only tree and run tests and then bounce back to the PiBox tree to integrate the changes?

Here’s another scenario:  I have a slew of metabuilds that do nothing but download 3rd party utilities, configure them as needed for PiBox and them package them as OPKG packages.  These builds are then wrapped in another metabuild that builds all of those package-metabuilds in order to simplify creating a complete PiBox release!  This top level metabuild has dependencies on the package metabuilds so I may find myself performing the top level metabuild, finding a bug (due to an upstream change to the branch I’m using)  in a 3rd party utility, switching over to that package metabuild to fix, commit and push it, and then bouncing back to the top level metabuild to rebuild the packages again.

What I need are common navigation tools for all projects and a way to change that navigation based on which project I want work on right at this moment.  The way to do that is with shell variables.  But I need to do this without having all kinds of project-specific shell scripts that are all different.  There needs to be some consistency in how they are used.  I want to set the project name but use the same navigation to end up in source, build and package trees no matter which project I’m in.


This shell script acts as a front end to a directory of configuration scripts, one for each project in which I do builds.  This script lives in ~/bin and sources any shell files found under ~/bin/env*.  In this way I separate builds for work from builds I do at home by placing the environment configurations in different directories, as in


Whenever I add a new project (and this happens frequently for me) I just drop in a new shell file in on of the env subdirectories and then reload cdtools.

. ~/bin/cdtools

This makes it easy to make changes without having to logout and back in.  It also makes it easy to do within a screen session so I can test new or modified configurations under one screen window without affecting the other windows until I’m ready for the update.

shell functions

Each project has its own shell file which contains a project specific shell function that sets up environment variables and aliases.   The environment variables are always the same but point to different things.  Typically I use variables prefixed with GM_, though not always.

  • SRCTOP – top of all source trees, under which you find all project specific directories.
  • GM_HOME – top of the project directory, under which you find the project source, build and package trees, along with others as needed.
  • GM_WORK – a scratch pad area often common between projects.
  • GM_BUILD – the build directory for the project
  • GM_SRC – the source directory for the project
  • GM_PKG – the packaging directory for the project

Other variables can be set to update the executable PATH, library path, Java path, or anything else that’s needed.  As long as the same variables are used in each shell file (with unused variables ignored) then switching from one project to another will always properly setup the environment.

For the use case, the PiBox shell function is rpi and the Linux kernel shell function is kernel.  Typing either configures my environment to access, build and navigate their respective trees.


Remembering the environment variables is hard.  So is typing them.  So the shell function maps them to navigation aliases.  All navigation aliases are prefixed with cd, as in

  • cdt – top of all source trees, under which you find project directories
  • cdh – top of the project directory
  • cdx – source tree under the project directory
  • cdb – build tree under the project directory
  • cdp – package tree under the project directory
  • cda – archive directory, where a local copy of downloaded source is kept, to avoid having to redownload unless absolutely necessary (this includes git or mercurial managed trees, as needed)
  • cde – Extras directory

A special alias, cd?, is setup to call a project shell function that lists the aliases, important environment variable configuration and help in cloning and accessing remote trees.  All projects use the same aliases.  Additional aliases can be setup as long as the context of the alias is the same for each project.

help functions

Until you’ve used cdtools for awhile it can be hard to remember the aliases.  It’s even harder to remember the sometimes convoluted git or mercurial (or other source code manager system)  commands necessary to work with remote repositories.  A help function is included in each shell file.  This is always prefixed with “list”.  In our use case, the helper function is listrpi or listkernel.  But you don’t have to remember this because the alias cd? always points to the currently active projects help function.

$ cd?
raspberrypi Alias settings:
cdt cd SRCTOP (/home/mjhammel/src/ximba)
cdh cd GM_HOME (/home/mjhammel/src/ximba/raspberrypi)
cdw cd GM_WORK (/home/mjhammel/src/ximba/work)
cdx cd GM_SRC (/home/mjhammel/src/ximba/raspberrypi/src)
cdb cd GM_BUILD (/home/mjhammel/src/ximba/raspberrypi/bld)
cda cd GM_ARCHIVE (/home/mjhammel/src/ximba/raspberrypi/archive)
cdp cd GM_PKG (/home/mjhammel/src/ximba/raspberrypi/pkg)
cde cd GM_EXTRAS (/home/mjhammel/src/ximba/raspberrypi/extras)
To checkout tree:
mkdir raspberrypi
git clone ssh:// src


So what happens when you have umpteen projects and you can’t remember which shell function sets up a particular project?  You can get a list of available functions and their descriptions using the cdlist function, which is included in the cdtools script.  The cdlist function browses all shell files and extracts the shell function name, a description and the name of the file its found in.  The description uses a tag in a shell comment, as in this example.

 # DESC: PiBox: Embedded environment for ARM-based system using buildroot

To search for this I can pass any string in the DESC field, such as PiBox:

 $ cdlist pibox
 pmsui PiBox Media Center UI (pms) []
 pmsuip PiBox Media Center UI (pms) []
 psp PiBox psplash opkg build []
 rpi PiBox: Embedded environment for ARM-based system using buildroot [raspberrypi]

All shell files with descriptions that include the string “pibox” (case insenstive) are listed.  Now I can see the shell function (rpi)  which is listed first and the file in which that function is defined (raspberrypi), which is listed at the end of the line inside brackets.

Example usage

First, load all cdtools scripts.

. ~/bin/cdtools

Now setup to work in the PiBox tree.  Then change to the source tree, see what’s there and then switch to the packaging tree and do the same.

 ls -C
 Makefile  README  configs  docs  pkg  scripts  src
 ls -C
 firmware opkg rootfs.ext3
 kernel.img.3.10.y opkg.10 rpiToolchain-0.9.0_Fedora19_201409081418-1.x86_64.rpm
 kernel.img.3.14.y opkg.9 rpiToolchain-0.9.0_Fedora19_201409162158-1.x86_64.rpm pibox-0.9.0_Fedora19_201409081418-src.tar.gz staging.tar.gz pibox-0.9.0_Fedora19_201409081418.tar.gz

I can see that I have done packaging after multiple builds here.  Wait, let’s check what’s in the top level metabuild tree.

 ls -C  docs  src

Okay, that looks fine.  Now back to the PiBox tree.


Easy peasy.

Multiple versions of a single repo

A neat trick here is that if you format the shell function correctly you can append a number.  That number will allow you to have multiple versions of the same repo, as in this example.

 rpi 2
 rpi 3

Checking the alias of the middle one, we see this.

raspberrypi Alias settings:
cdt cd SRCTOP (/home/mjhammel/src/ximba)
cdh cd GM_HOME (/home/mjhammel/src/ximba/raspberrypi2)
cdw cd GM_WORK (/home/mjhammel/src/ximba/work)
cdx cd GM_SRC (/home/mjhammel/src/ximba/raspberrypi2/src)
cdb cd GM_BUILD (/home/mjhammel/src/ximba/raspberrypi2/bld)
cda cd GM_ARCHIVE (/home/mjhammel/src/ximba/raspberrypi2/archive)
cdp cd GM_PKG (/home/mjhammel/src/ximba/raspberrypi2/pkg)
cde cd GM_EXTRAS (/home/mjhammel/src/ximba/raspberrypi2/extras)
To checkout tree:
mkdir raspberrypi2
git clone ssh:// src

Note the “2” in the directory paths.  I can now have multiple copies of the same repo.  And this trick can go further by using additional arguments to the shell function to embed different-yet-related repositories using a single shell function.

 bui fakekey
 bui network-config 2

And so forth.


The cdtools script currently includes support for color coding of the cdlist using ASCII escape sequences.  This works in most terminal types.  More importantly, there is no reason you can’t extend cdtools to support additional functionality that you want to embed in your own set of shell files.

Consistency of practice

What you should get out of this is the consistency of use when moving from one project to another.  How I build projects can vary greatly: PiBox uses GNU Make while one of its packages (launcher, for example) uses Autoconf.  This happens all the time because when doing a system build (a build of an OS plus utilities plus custom kernels plus custom apps, etc.) you may depend on many 3rd party source trees with a variety of build patterns.  But that doesn’t matter.  What matters is can I build it quickly by bouncing to it and having pretty much everything I need in place when I do.

With cdtools, the answer to that question is a resounding YES.

Related posts

Networking with QEMU and KVM speedup

[Print This] By mjhammel ~ April 6th, 2015. Filed under: Fedora, Hardware, Linux, virtualization.

Networking in QEMU

I’ve recently had a need to test UEFI booting for a disk image.  I stumbled upon OVMF, a bios that will handle UEFI booting.  It works pretty well but doesn’t remember its config sometimes.  But that’s not why I’m writing this.

In my disk image I need dual network interfaces that are on specific networks.  I don’t need tap interfaces.  I can live with the slower user-mode interfaces.  But I needed to put them on separate networks.  So I did this.

-netdev user,id=user0,dhcpstart=,hostfwd=tcp::5555-:22 -device e1000,netdev=user0
-netdev user,id=user1,dhcpstart=,hostfwd=tcp::5556-:22 -device e1000,netdev=user1

I googled till I was blue to figure out how to do this.  You can set the dhcpstart address but qemu_system_x86-64 spews the following message:

device 'user' could not be initialized

It doesn’t say why, but after fiddling with it for a long time it became obvious that this was a configuration error.  There just doesn’t seem to be any explanation as to what configuration is wrong.

Finally I found a site with an example.  The key appears to be that you have to include the host’s network address on the network that interface will be in.  Without this specification there is no mapping of the guest interface to a host interface.  Here is what the correct configuration should look like.

-netdev user,id=user0,net=,dhcpstart=,hostfwd=tcp::5555-:22 -device e1000,netdev=user0
-netdev user,id=user1,net=,dhcpstart=,hostfwd=tcp::5556-:22 -device e1000,netdev=user1

Now both interfaces will come up on the assigned networks.  The hostfwd option is used to map host ports to guest ports, with the larger number (the first one) mapping to the guest’s port.  Here I’m mapping ports to the ssh port on the guest.  Since I’m using user-mode networking this is required otherwise the default firewall in the guest prevents me from accessing it from the host.

KVM Speedup

Another thing I discovered when running qemu_system_x86_64 is that it was slow on my Fedora 21 box.  The reason is simple:  you can’t run this command manually with KVM speedups (which are significant) if you’re already running libvirtd.  You can go through the hoops of tying the two together, but for a quick solution to run a qemu session with full virtualization to get much better performance without dealing with libvirtd just shutoff libvirtd.

sudo service libvirtd stop

That’s using archaic service format instead of the ugly sysctl interface (that you very little, systemd) but it works.  Again, this is for Fedora.  Your distro mileage will vary.

Related posts

Migrating from gitorious – it sucks.

[Print This] By mjhammel ~ April 1st, 2015. Filed under: General.

I can no longer push to gitorious so I tried to run the import of all my repos (and there are alot) into GitHub. This failed miserably. Only two of 34 repos were imported. Some of the repos actually got created on GitLab but there is nothing in them. So I tried to import one of them manually. It’s been running for about 20 minutes now and nothing is happening. No errors, no messages, no nothing. Just a spinning wheel.

I dug around and found the process for migrating a repo, be it to GitLab or GitHub or whatever.  It requires manual creation of each repository.  While gitorious allowed me to have different projects with different repos under them, GitHub does not.  At least I don’t see a way to do that.  GitLab does, but they call them groups.  The import appears to have created the groups but that’s all it did.

So I guess I can try doing this manually under each group, after manually creating each repo.  *sigh*  This is a strong argument for running my own git server and gitweb.

Related posts

apache 2.4/Fedora quirk in welcome.conf

[Print This] By mjhammel ~ March 10th, 2015. Filed under: Fedora, Linux, Software Development, Ubuntu.

I was trying to set up a directory index via a named vhost under Apache today.  Typically all you need to do for this scenario is something like this in an apache conf file.

<VirtualHost *:80>
DocumentRoot /home/httpd/files
<directory "="" home="" <span="" class="hiddenSpellError" pre="" data-mce-bogus="1">httpd/files">
Options +Indexes

Then restart apache and make sure the server name resolves somehow (DNS or /etc/hosts) and you’re good to go.

But with Apache 2.4 something changed.  This doesn’t work.  You get the nice welcome page that says the web server is running but you don’t get the directory index.  I found a solution online that hid the solution behind a requirement to “like” the answer but it became very clear what the problem was: the welcome.conf configuration file overrides my directory index.

In welcome.conf you find this bit of code.

<LocationMatch "^/+$">
Options -Indexes
ErrorDocument 403 /.noindex.html

That Options -Index line is the problem.  The easiest solution, in this case where all I want is a web server that shows a directly listing, is to comment out that whole block of code.  Restarting the web server and accessing my file server works as expected now.

I had this problem on Fedora 21.  I’ve seen others have it with Ubuntu.  So the source of welcome.conf content may be the apache project itself.  In other words, this solution may be applicable to any distribution using Apache 2.4 or later.

Related posts