Version 36 (modified by 7 years ago) (diff) | ,
---|
Vision Framework
Objective
Create a reprogrammable module in which a Raspberry Pi 3, protective case, power cable, light ring, and camera are self contained and execute a base program on startup. The base code of the module continuously reads the camera feed (subscribes to camera) and publishes corresponding data via the Pi's serial output (located on the Pi's GPIO connector). This data can then be fairly simply read by the RoboRIO. We hope that this will make implementing a vision subsystem a more simple proposition, allowing more teams to do so.
Materials
- Clear Case
- Raspberry Pi 3
- Micro SD card
- Usb Keyboard
- Usb Mouse
- HDMI cable
- Monitor (that accepts HDMI input, or use an HDMI to VGA adapter)
- Ribbon Cable Camera
- Micro USB Cable
- Adafruit NeoPixel? Light Ring
- 3 individual male to female PWM wires
- 3 individual female to female PWM wires
Raspberry Pi Setup
Raspbian Lite
Install Win32 Disk Imager
Install Raspian Lite onto MicroSD card via Win32 Disk Imager
Load Raspberry Pi 3 (rpi) with the newly imaged MicroSD
username: pi password: raspberry
connect to wifi by adding the following to the /etc/network/interfaces file:
auto wlan0 iface wlan0 inet dhcp wpa-ssid "your-ssid" wpa-psk "your-password"
reboot the pi:
sudo shutdown -r now
update with the following commands:
sudo apt-get update sudo apt-get upgrade sudo apt-get dist-upgrade sudo apt-get clean
set localization configuration options:
sudo raspi-config
reboot:
sudo shutdown -r now
X server
Install Xorg and Xinit:
sudo apt-get install --no-install-recommends xserver-xorg sudo apt-get install --no-install-recommends xinit
Install the MATE GUI:
sudo apt-get install mate-desktop-environment-core
Install LightDM login manager:
sudo apt-get install lightdm
reboot:
sudo shutdown -r now
login to MATE
At the top left:
navigate to preferences -> hardware -> keyboard shortcuts
to set Run a Terminal to Ctrl + Alt + T
or the like
open a terminal
navigate to Edit -> Profile Preferences -> Colors
to deselect Use colors from system theme
select Built-in schemas: White on black
Development Tools
Java 8
Now that the terminal color scheme is not killing you, we are going to start off by installing java:
sudo apt-get install oracle-java8-jdk
test the java installation (output should include "1.8.0_65" and should not include "openjdk"):
java -version
Let's also set the JAVA_HOME variable to be exported on startup:
echo "export JAVA_HOME=/usr/lib/jvm/jdk-8-oracle-arm32-vfp-hflt" >> ~/.bashrc source ~/.bashrc
Finally, let's install ANT:
sudo apt-get install ant
Pi4J (Java-GPIO Interface)
Now we can install pi4j, a Java interface for the pi GPIO:
curl -s get.pi4j.com | sudo bash
rpi_ws281x (NeoPixel? Control Library)
Then we can use the rpi_ws281x library for controlling the Adafruit NeoPixel lightring
But first, let's install some tools and dependencies:
sudo apt-get install build-essential python-dev git scons swig
Now we can install the rpi_ws281x library:
git clone https://github.com/jgarff/rpi_ws281x.git cd rpi_ws281x scons
OpenCV
First of all, order some pizza or something because this is going to take about 3 hours (if you are lucky).
EDIT: we have tried with a heatsink as well. It doesn't work. You need to water cool your pi. Please skip to the WATER COOLING section.
Also, you need a heatsink for your processor before building OpenCV. If you see this overheating symbol, you will die. Straight up.
If you don't have a heatsink on hand, tape some pennies to the CPU like we did:
================WATER COOLING================
If this still doesn't work, it may be necessary to water cool your pi:
Make sure to change the ice cube at least once every half hour. Also, only single bag the ice cube.
Now we are going to install OpenCV and its dependencies.
We only install OpenCV for C++ and Java, not for Python (we ain't scrubs):
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev cd git clone https://github.com/opencv/opencv.git cd opencv mkdir build cd build cmake .. make -j $[$(nproc)+1] sudo make install
This builds a single shared object file (libopencv_java320.so), a jar file (opencv-320.jar) and a ton of static libraries (*.a)
It copies the shared object file and the jar file to the install directory: /usr/local/share/OpenCV/java/...
NOTE: we could not get the java wrapper to work with ffmpeg enabled (we ran into a Segmentation fault with opencv libs 3.2.0, 3.3.0, and 3.3.1). We gave up and disabled it.
If you do not want to git clone the master opencv repository, download a zipped version:
opencv-3.2.0.zip
Java Projects with OpenCV
Enable VideoCapture?
First, to allow OpenCV VideoCapture? to access the ribbon cable camera, execute the following:
echo "sudo modprobe bcm2835-v4l2" >> ~/.bashrc source ~/.bashrc
Compiling
javac -d <build_path> -classpath <external_jars> <source_path>/*.java
where:
build_path = directory to store class files
external_jars = string specifying individual jar dependencies (such as OpenCV and Pi4J) with delimiters as follows:
:/path_to_jars/jar1.jar:/path_to_jars/jar2.jar
souce_path = directory containing all of your java files to be compiled
We recommend creating a libs
folder in which you store all of your external jars such as the opencv-320.jar
and all the Pi4J jars rather than having your libraries scattered all over your machine.
Running
sudo java -Djava.library.path=<opencv_install_path> -Dpi4j.debug -Dpi4j.linking=dynamic -classpath <external_jars> <Main_class>
where:
external_jars = same string that you used in the Compilation step
Main_class = name of the class file with main method
- For example, where
Vision.class
is the class file,Main_class = Vision
- For example, where
C++ Projects with OpenCV
NOTE: our performance doubled when we transitioned to C++ from Java
With C++, we can process (capture and segment) 640x480 frames at 66 fps and 1080x720 frames at 28 fps
With Java, we could only get about 30 fps with 640x480 frames
Let's install an IDE so we do not have to keep writing in vi/vim/nano:
sudo apt-get install codeblocks
We experienced a bug that constantly crashed CodeBlocks:
To prevent crashing, navigate to: Settings -> Editor -> Code completion -> Symbols browser
and check Disable symbols browser
(the Code completion
tab is on the west panel of the configure editor.)
Navigate to: File -> New -> Project -> Console Application
And fill in the fields that come up.
If your Management
tab is not visible, open it by selecting: View -> Manager
Also, if your Logs & others
tab is not open already, select View -> Logs
Now right click your project in the Management
tab and navigate to: Build options... -> Linker settings
Under the Link libraries
panel, select Add
and select all OpenCV shared object (.so) files located in /home/pi/opencv/build/lib/*.so
(You can [ctrl + left-click] multiple *.so] files to add them all at once)
Now let's navigate to: Search directories -> Linker
In the Linker
tab, select Add
and add the path to your opencv lib: /home/pi/opencv/build/lib
Tricks and Tips
Using OpenCV with the Raspberry Pi 3 ribbon cable camera is a little cumbersome because you cannot use many of the VideoCapture?.set(char*,int) functions.
Instead, we use system commands to set camera settings via the v4l2-ctl library.
For FRC retroreflective tape segmentation, use the following system commands to prevent auto adjustment features such as auto exposure
, white balance
, and exposure time
:
- v4l2-ctl --set-ctrl=auto_exposure=1
- v4l2-ctl --set-ctrl=white_balance_auto_preset=0
- v4l2-ctl --set-ctrl=auto_exposure_bias=0
- v4l2-ctl --set-ctrl=exposure_time_absolute=100
Make your C++ program set these options by using the stdlib.h
system(char*)
command which simply takes a string command as input.
Alternatively you could add these options to your bashrc
file like we did with the modprobe, but you may want to tweak these options or experiment with them, so putting them into your C++ program makes it a little easier to manipulate.