PyBBIO update – version 0.9.3

Version 0.9.3 of PyBBIO is out tonight, which includes some great performance upgrades and a new LCD library.

GPIO Speed

People tend to worry about the speed of GPIO operations when using sysfs entries. While it's in many ways arbitrary for timing sensitive applications because no GPIO access from userspace is real-time anyway, version 0.9.3 brings significantly faster GPIO operations. All of the GPIO related code is now in C extensions, and all GPIO state file descriptors used in a program are kept open until it exits.

Running this test program with v0.9.2:

from bbio import *
pinMode(GPIO1_28, OUTPUT)
while 1:
  digitalWrite(GPIO1_28, HIGH)
  digitalWrite(GPIO1_28, LOW)

The pin is toggled at about 1.5kHz. Running the same program with v0.9.3 now toggles the pin at about 25kHz, or around 17x faster.

LiquidCrystal

There's also a new library for driving HD44780 compatible character LCDs. It supports the majority of the LCD's functionality (though not all of it is tested yet!), including scrolling and custom glyphs. API docs coming soon...

--

The rest of the major changes are:

  • Moved all GPIO code to C extension
  • Added LiquidCrystal character LCD library
  • Improved sysfs interface for faster kernel driver file access
  • Removed 3.2 support, use 0.9.2 if you're still running 3.2 for some reason...
  • Libraries are now contained within the bbio package. This changes importing a bit, user code will need updating
  • Simplifies setup.py
    • Examples are no longer copied
    • DT overlays are distributed compiled and copied with setuptool as data_files
  • BBIOServer now serves from ~/.BBIOServer instead of from inside the package
  • Started adding stubs for universion-io support, not yet implemented

PyBBIO update – version 0.8.5

This is really the first official PyBBIO update, but I'll try to make it a more regular thing.

GPIO

With PyBBIO's update from v0.8 to v0.8.5 comes the elimination of the mmap based approach for GPIO control.

Why do this? Because going behind Kernel drivers' backs is just plain wrong! When I first started working on PyBBIO the BeagleBone kernel was in a very different state, and most of the IO was done through hacks of one sort or another. But with the introduction of the Device Tree to the BeagleBone with Kernel 3.8, and thanks to all the great driver development from TI and other BeagleBone users, it now makes much more sense to take a fully Kernel driver based approach as I move forward with PyBBIO.

If you're wondering whether this change will slow down PyBBIO's GPIO, the answer is yes, by orders of magnitude! But if you need high speed GPIO, you really shouldn't be controlling it from user-space Linux. In fact, if you want high-speed GPIO on the BeagleBone, that's what the PRUSS is for! (Some of my notes and resources on using the PRUs is on Github here)

The mmap based version is still available (without any support or further development) on its own mmap-gpio branch.

I've also fixed the USR LEDs, which hadn't been working for quite some time (oops!).

PWM

There's now six PWM outputs working on Kernel 3.8: the 4 ePWM outputs that were previously working, plus eCAP0 and eCAP1. (ECAP0 and ECAP1 variables in PyBBIO)

I2C

Deepak Karki contributed a great I2C library, thanks Deepak! It provides the two objects Wire1 and Wire2, which have an interface similar to the Arduino Wire library. Documentation to come. (source code here)

Device Tree

With Kernel 3.8 came the Device Tree, which was a pretty major speed bump for user-space IO libraries because it completely changed the interface to enable and configure the AM3359's subsystems.

The way PyBBIO handles this is to dynamically create small and very specific Device Tree overlays during installation. These include separate overlays for each GPIO pin, for example. My reasoning for taking this approach, rather than having a single huge PyBBIO overlay, is that it means PyBBIO is inherently compatible with cape overlays because it only uses the resources you explicitly tell it to.

Coming soon

PyBBIO is part of GSoC 2014, and Rekha Seethamraju will be working all summer to add cool new stuff! Updates will be posted here as she goes.

BeagleBone pinout

I've finally gotten around to drawing up a pinout for the BeagleBone's headers. There's a few floating around the internet, but I wanted one specific to PyBBIO. It shows all the pins currently supported by PyBBIO, color coded by type, and each with the same name as its pin constant.

BeagleBone pinout


- Update 1/11/13: added all GPIO pins available on headers - Update 1/17/13: fixed Serial5 TX/GPIO0_3 pin - Update 6/14/13: fixed a few GPIO pins



I made this using Inkscape. For anyone who wants it you can download the .svg file here.

BeagleBone’s SETDATAOUT and CLEARDATAOUT registers

I recently received an email from Mark Stephens (mrmorphic on github) pointing out that I could be using the AM3359's GPIO_SETDATAOUT and GPIO_CLEARDATAOUT to set the state of the BeagleBone's GPIO pins in the digitalWrite() routine of my PyBBIO library (Mark has been working on a similar library in Go at github.com/mrmorphic/hwio). I had previously been setting pin states the way I'm used to doing it on AVR and MSP430 microcontrollers, which is to grab the entire pin state register (GPIO_DATAOUT on AM335x chips), which might look something like this:

00000010 00000000 00100000 00000000

Then create an equal length value of all 0s with the bit corresponding to the desired pin set to 1, e.g. if setting the state of pin 7 in the register it would 1<<7, or:

00000000 00000000 00000000 10000000

Then to set the pin high, the GPIO_DATAOUT register is set to its previous value bitwise OR-ed with the new value:

   00000010 00000000 00100000 00000000
OR 00000000 00000000 00000000 10000000
   -----------------------------------
 = 00000010 00000000 00100000 10000000

And to then set the pin low, the GPIO_DATAOUT register is set to its value bitwise AND-ed with the inverse, or twiddled version, of the new value:

 ~ 00000000 00000000 00000000 10000000
   -----------------------------------
 = 11111111 11111111 11111111 01111111

    00000010 00000000 00100000 10000000
AND 11111111 11111111 11111111 01111111
    -----------------------------------
  = 00000010 00000000 00100000 00000000

The issue with using this method in PyBBIO is that packing and unpacking 32-bit memory registers in Python is a relatively slow process. Luckily the AM335x MPUs provide two additional registers for manipulating the state of digital output pins: GPIO_CLEARDATAOUT and GPIO_SETRDATAOUT, which are used to set output pins low and high respectively. As soon as a 1 is written to any of the bits in one of these registers, the according pin is set high or low, and the bit is set back to 0. So writing a 0 to a bit in one of these registers will have no effect, because that is the normal state of all the bits. This means that the SETDATAOUT register can only set pins high, and the CLEARDATAOUT register can only set pins low, and we no longer have to worry about maintaining register states. Using these registers in digitalWrite(), we can cut our register access in half.

To see how much of an improvement this change actually makes, I wrote a quick program to toggle the state of a GPIO pin as fast as possible:

from bbio import *

def setup():
  pinMode(GPIO1_6, OUTPUT)

def loop():
  state = 1
  while(True):
    digitalWrite(GPIO1_6, state)
    state ^= 1

run(setup, loop)

I then hooked my scope and ran the program with digitalWrite() not using the SETDATAOUT and CLEARDATAOUT registers, and I clocked it at around 10.6KHz (frequency is in the bottom right of the image):

PyBBIO_digitalWrite_slow

Then I made the change to digitalWrite() and ran the program again, and low and behold it just about doubled the speed of the toggling to just under 20KHz:

PyBBIO_digitalWrite_improved

BeagleBone IO using Python and mmap

The BeagleBone runs an Angstrom linux distro, which includes the special /dev/mem file, which provides access to the processor's entire physical memory. More on /dev/mem in the linux man pages here. The BeagleBone is built around the TI AM3359 ARM Cortex-A8 microprocessor, and breaks out many of its I/O pins to two 2x23 pin female headers, including 4 UART ports, 7 ADC inputs, 4 PWM outputs, a whole bunch of GPIO pins and more. The use of these peripherals is very well documented in the AM335x Technical Reference Manual. The use of each module is done through reading and writing to different 16- and 32-bit memory registers, whose addresses are given in the manual. A Unix mmap can be used to create a memory map of the /dev/mem file, which allows read and write access to it through byte addressing. This way the module registers may be accessed by their addresses in section 2.1 of the reference manual.

The most basic module to use is the GPIO module. Other modules get more complicated, but they are all very well documented in the Technical Reference Manual. The biggest difference is that many of the other modules have clocks which must be enabled before they can be used. This is done using the clock module registers, and is described in detail in chapter 8.

I'll walk through blinking one of the on-board LEDs using Python's mmap module. Before starting, we need to know which pin to use. To do so, we want a copy of the BeagleBone schematic handy, which can be found in the links here or on the uSD card supplied with the BeagleBone. According to page 3 of the schematics, the USR1 LED is connected to the GPIO1_22 pin (easily found by searching 'USR1' in the pdf), which means pin 22 in the GPIO1 module. Now it's time for a little Python. First we need to import the mmap module, which is not on the BeagleBone by default, but can be installed by running '# opkg install python-mmap', and defining a few addresses:

from mmap import mmap
import time, struct

GPIO1_offset = 0x4804c000
GPIO1_size = 0x4804cfff-GPIO1_offset
GPIO_OE = 0x134
GPIO_SETDATAOUT = 0x194
GPIO_CLEARDATAOUT = 0x190
USR1 = 1<<22

Mapping the entire /dev/mem file would require that over a gigabyte be allocated in Python's heap, so the offset address and size variables are used to keep the mmap as small as possible, in this case just the GPIO1 register. These values are straight out of the memory map in section 2.1 of the Technical Reference Manual. the GPIO_OE, GPIO_SETDATAOUT and GPIO_CLEARDATAOUT addresses are found in section 25.4, which shows the address offsets of each register within the GPIO modules, starting from the base module address. Chapter 25 explains how to use the GPIO registers. All we need to do is set a pin as an output, then set and clear its output state. To do the first, we need the 'output enable' register (GPIO_OE above). Then the GPIO_SETDATAOUT and GPIO_CLEARDATAOUT registers will do the rest. Each one of these registers is 32 bits long, each bit of which corresponding to one of 32 GPIO pins, so for pin 22 we need bit 22, or 1 shifted left 22 places.

Next we need to make the mmap, using the desired size and offset:

with open("/dev/mem", "r+b" ) as f:
  mem = mmap(f.fileno(), GPIO1_size, offset=GPIO1_offset)

Now we need to set the pin as an output. I'll do this line by line:

The mmap is addressed byte by byte, so we can't just set a single bit. The easiest thing to do is grab the whole 4-byte register:

packed_reg = mem[GPIO_OE:GPIO_OE+4]

We now have 32 bits packed into a string, so to do any sort of bitwise operations with it we must unpack it:

reg_status = struct.unpack("<L", packed_reg)[0]

The 'L' tells struct.unpack() to unpack the string into an unsigned long, which will give us the full 32-bit register. The '<' tells it that the string is packed little-endian, or least-significant byte first. The BeagleBone's memory is little-endian, so if we tell this to struct.unpack() it will return the 32 bits in the order they are shown in the reference manual register maps.

We now have the 32-bit integer value of the register, so we can configure the LED as an output by clearing its bit:

reg_status &= ~(USR1)

Now all that's left to do is to pack it little-endian back into a string and update the mmap:

mem[GPIO_OE:GPIO_OE+4] = struct.pack("<L", reg_status)

Now that we know the pin is configured as an output, it's time to get blinking. We could use the GPIO_DATAOUT register to do this, but we would want to preserve the state of all the other bits in it, so we would need to do the same process of unpacking, manipulating then repacking. That's what the SETDATAOUT and CLEARDATAOUT registers are for. Writes to them affect only the pins whose bits are set to 1, making the next step much easier:

for i in range(5):
  # Set it high:
  mem[GPIO_SETDATAOUT:GPIO_SETDATAOUT+4] = struct.pack("<L", USR1)
  time.sleep(0.5)
  # Set it low:
  mem[GPIO_CLEARDATAOUT:GPIO_CLEARDATAOUT+4] = struct.pack("<L", USR1)
  time.sleep(0.5)

mem.close()

Which will blink the LED 5 times then close the mmap and exit.

There is an issue here which is not at all obvious, which is that setting a pin as an output in the output enable register in no way guarantees that the corresponding physical pin will be connected to it. In fact, there is no guarantee that any pin will be as it is labeled in the schematics! This is due to the fact that the AM335x processors have a whole lot more module inputs and outputs than there are physical pins, so every external pin is actually the output of a multiplexer with up to 7 different possible modes. All this pinmuxing is handled by the AM335x control module. Of course there's a catch, which is hiding in section 9.1:

Note: For writing to the control module registers, the Cortex A8 MPU will need to be in privileged mode of operation and writes will not work from user mode.

Luckily, thanks to the friendly BeableBone developers, there is a user-level workaround. There is a file for each external pin found in /sys/kernel/debug/omap_mux/. Writing to these files tells a driver to configure the pin multiplexers as desired. To find the proper file names is a bit of a pain, and requires one more document; the AM3359 datasheet, found here. The first step is to find the pin of interest in the Beaglebone schematics. The USRx LED outputs actually are muxed correctly when the board boots, so for this example I'll choose a different GPIO pin. Lets say we want to use GPIO2_7, which is pin 46 on the P8 header, as found on page 11 of the schematics. We need to find where the pin is connected to the AM3359, which is easily done by searching 'GPIO2_7'. What we're after is the long list of modes for the pin, which in this case is:

LCD_DATA1/GPMC_A1//EHRPWM2B//PR1_PRU1_PRU_R30_1/PR1_PRU1_PRU_R31_1/GPIO2_7

Now we open up the AM3359 datasheet and go to Table 2-7. What we're looking for is the pin for which the first item in this list, in this case LCD_DATA1, is the first item in the 'signal name' column. Notice if you use the search function to find it, it shows up a few times in the 'signal name' column. These are different pins which can also be set to this function, but they are not the actual physical pin we are looking for. Once found, look down the 'signal name' column within the LCD_DAT1 row until you find GPIO2_7, and take note of the mode number just to the right of it (7 in this case). Also take note of the 'pin name' for mode 0, as this will be the name of the file we need to write to, in this case 'lcd_data1' (these are all upper-case in the schematics, but the file names are all lower-case). While here, take a look at the 'buffer strength' column. This is the maximum current the pins can source; only 6mA for the GPIO pins! This is usable for low-power LEDs and the such, but external transistors are generally a must for any sort of power supply switching.

Now it's back to the reference manual for a minute, to Table 9-58 in section 9.3.1.50. This shows us everything we can configure with the control module files, including pullup and pulldown resistors, and 'input enable'. The input enable has no effect on output behavior, so it can just be left set for the GPIO pins. So, what we want is to set the LCD_DAT1 multiplexer to mode 7 with rx_enabled. The pinmux files expect the desired value to be written in hexadecimal, so the binary value we want of 100111 (1<<5 for receiver enable | 111 for mode 7) must be written as its hex value of 27. This can be done at the command line:

# echo 27 > /sys/kernel/debug/omap_mux/lcd_data1

Or in Python:

with open("/sys/kernel/debug/omap_mux/lcd_data1", 'wb') as f:
  f.write('27')

And we can easily confirm the change with:

# cat /sys/kernel/debug/omap_mux/lcd_data1

So to show the whole process, here's a script that will blink an LED connected to GPIO2_7 until ctrl-c is pressed (notice that the mmap offset has changed, as the GPIO pin used here is in the GPIO2 module, where as we were using GPIO1 before):

from mmap import mmap
import time, struct

GPIO2_offset = 0x481ac000
GPIO2_size = 0x481acfff-GPIO2_offset
GPIO_OE = 0x134
GPIO_SETDATAOUT = 0x194
GPIO_CLEARDATAOUT = 0x190
LED = 1<<7

with open("/dev/mem", "r+b" ) as f:
  mem = mmap(f.fileno(), GPIO2_size, offset=GPIO2_offset)

with open("/sys/kernel/debug/omap_mux/lcd_data1", 'wb') as f:
  f.write('27')

reg = struct.unpack("&lt;L", mem[GPIO_OE:GPIO_OE+4])[0]
mem[GPIO_OE:GPIO_OE+4] = struct.pack("<L", reg & ~LED)

try:
  while(True):
    mem[GPIO_SETDATAOUT:GPIO_SETDATAOUT+4] = struct.pack("<L", LED)
    time.sleep(0.5)
    mem[GPIO_CLEARDATAOUT:GPIO_CLEARDATAOUT+4] = struct.pack("<L", LED)
    time.sleep(0.5)

except KeyboardInterrupt:
  mem.close()

I've been working on a Python library to make the tedious process above as painless as possible, which can be found at github.com/alexanderhiam/PyBBIO. As of writing this, it includes a simple API for the GPIO modules, the ADC, and the UART serial ports, as well as a few built-in libraries, like one for building simple web interfaces to control hardware IO. To show the difference, here's the same script above, but this time using PyBBIO:

from bbio import *

LED = GPIO2_7

def setup():
  pinMode(LED, OUTPUT) # This does the pinmuxing automatically

def loop():
  toggle(LED)
  delay(500) # In milliseconds

run(setup, loop)

Super Simple BeagleBone Web Interface!

I've been working for a while now on PyBBIO, my Python library for BeagleBone hardware expansion. It's got some neat features, but my favorite so far is the BBIOServer library, which makes creating web interfaces to PyBBIO incredibly simple.

BBIOServer provides two things: a web server class called BBIOServer, and a HTML page class called Page. To use it you create a Page instance for every page you want your web interface to have, add content to them through the Page methods, then pass them all to the start() method of a BBIOServer instance, and it will create the pages complete with a sidebar to navigate between them, then start the server on the default port of 8000, or whatever port you have specified.

To create a single page with a button that toggles the USR3 on-board LED, as well as a monitor of its current state, would look something like this:

from bbio import *
from BBIOServer import *

server = BBIOServer()

def setup():
  page = Page("Server test")
  page.add_text("This is a test of the BBIOServer library.")
  page.add_button(lambda: toggle(USR3), "Toggle USR3", newline=True)
  page.add_monitor(lambda: pinState(USR3), "Current state:")
  server.start(page)
  stop()

def loop():
  pass

run(setup, loop)

And if you run that and point your browser to http://your_beaglebone_ip_address:8000 you should see something like this:

bbioserver

In this case the Page instance uses the default stylesheet, but you can also point to other stylesheets in the PyBBIO/libraries/BBIOServer/src/ directory. Besides the default stylesheet, BBIOServer includes a stylesheet that makes the pages get along well with mobile browsers. If you replace page = Page("Server test") in the above code with page = Page("Server test", stylesheet="mobile.css"), then head to http://your_beaglebone_ip_address:8000 on your phone, you should see something like this:

bbioserver_mobile

Here's a quick video of one of the BBIOServer examples included in PyBBIO (see the code here):

Be sure to check out the BBIOServer documentation.