How To Set Up a SECN Build Environment
Author: Terry Gillett
- 1 Introduction
- 2 Scripted Setup of a Build Environment
- 3 Setting up a Build Environment Manually
- 4 Install Git and Subversion
- 5 Download the OpenWrt sources
- 6 Set up Package Feeds
- 7 Download Feeds
- 8 Install additional packages
- 9 Check for missing Ubuntu system packages
- 10 Set up the Device Configuration
- 11 Add Custom Files
- 12 Proceed with the Build
- 13 Collecting the output firmware files
- 14 Notes
This HowTo document is a simplified description of the process for setting up an OpenWrt build environment for producing firmware for VillageTelco SECN devices.
There are two parts to this document. The first part describes how to set up a build environment using pre-prepared Setup scripts, and the second part describes how to set up the build environment manually step by step. The end result of the two different processes is essentially the same.
The script based setup is designed reproduce the published SECN firmware, using the same versions of all the software repositories used to build the original firmware. It is based on the newer Git based source repositories.
Setting up the build environment manually will allow you to further customise the build process, and potentially choose different versions of the component software repositories. The manual process described here is based on the older Subversion based software repositories.
References: OpenWrt Wiki » Documentation » HOWTOs » OpenWrt Buildroot – Installation
Scripted Setup of a Build Environment
At the time of writing, Setup scripts using the Git based repositories are available for SECN 3 firmware, based on OpenWrt BB 14.07, and for SECN 4 firmware, based on OpenWrt CC 15.05
Once the Setup script has been run to create the build environment, then the Build scripts are used to build firmware for specific devices.
The VT SECN source code is held in the GitHub repository at:
Source code for SECN3 (based on OpenWrt BB) and SECN4 (based on OpenWrt CC) is held in the "secn" branch. You will need to check out the appropriate branch as below to get the required Setup script.
The Build scripts will automatically check out the appropriate branch when they are run.
For the purposes of this document, we assume that you will have a directory "~/Git" that will contain the local Git repositories, and a directory " ~/openwrt" that will contain the build environments. The Build scripts assume this directory structure by default, so if you change it you will have to amend the Build scripts accordingly.
You need to have installed the necessary Git utilities as follows:
$ sudo apt-get update $ sudo apt-get install -y git-core subversion build-essential gawk zlib1g-dev python
Create a local copy of the vt-firmware repository using commands as follows:
$ mkdir ~/Git $ cd ~/Git $ git clone https://github.com/villagetelco/vt-firmware
This will create a directory called "vt-firmware".
To set up a build environment you need to checkout the appropriate branch of the repo as follows:
$ cd ~/Git/vt-firmware For SECN3, 4 $ git checkout secn For SECN 5 $ git checkout secn5
If you look in the vt-firmware directory you will see a directory called "Setup-scripts" which contains the file "Setup-14.07.sh"
Copy this file into the "~/openwrt" directory, make it executable, then run it as below. The script will download several hundred MB of data so make sure you have a good Internet connection set up before you start.
$ cd ~/openwrt $ chmod +x Setup-14.07.sh $ ./Setup-14.07.sh ~/openwrt/Build-14.07
This will create a new directory called "Build-14.07" and download the required software sources. Various progress messages will be seen on-screen, and the details saved in log files for debugging if necessary.
The Setup script defines particular revisions of the OpenWrt source code and of the various source code repositories (including packages, telephony, routing, fxs) so that the resulting firmware builds will have the same software configuration as the published SECN firmware.
You can change the defined repository revisions by editing the "feeds.conf.default" file and running the command "$ ./scripts/feeds update -a"
However, be aware that changing the repository revisions may result in incompatibilities between packages and/or the underlying OpenWrt version, so proceed with caution.
Using the Build Scripts
Look in the "~/Git/vt-firmware" directory and you will see a folder called "Build-scripts". There are build scripts for MP02, TP-Link and Ubiquity devices.
Copy the appropriate device build script (eg Build-MP02.sh) into the new "Build-14.07" directory.
Edit the script and look at the beginning of the script to find the version strings that will be used to name the firmware files to be produced, and edit if/as required.
When the firmware is built, it will have a unique build date/time stamp included internally in the command line splash page, and in the directory name, so that individual firmware can always be distinguished, even if the version strings are the same. Be aware that if you build the exactly same firmware again, it will have a different md5sum due to the internal date stamp.
Look at end of the Build script to see the list of devices that will be built. Comment out the lines for devices that you do not require.
Then change to the build directory and run the build script as follows:
$ cd ~/openwrt/Build-14.07 $ ./Build-MP02.sh
The initial build will take quite quite a long time and will download a lot of data, so make sure you have a good, reliable Internet connection. The directory may grow to about 5GB in size so make sure there is adequate disc space available.
Subsequent builds will be considerably faster as the various component packages will have already been built.
The build script has optional lines for the main "make" command to take advantage of multi-core CPUs on your PC (eg make -j5 for a four core machine). It is advisable to run it with the default ( make -j1 ) for the initial build. While this is slower, it ensures that the build has the best chance of completing successfully.
For subsequent builds you can comment out this line and un-comment the multi-core line. Be aware that using the multi-core command may produce random build errors.
This "make" command also has a the flag "V=s" set by default. This provides a verbose output which is also saved in a log file so that you can look through it in the event that the build fails for some reason. Again, this is useful to have in place for the initial build.
One reason that builds can fail is that a particular set of software is temporarily not available for download from the Internet. This will be flagged at the end of the build log so you can see what has happened.
When the build completes successfully, the firmware and md5sums files will be saved in the "Builds" directory, and in a subdirectory according to the device type, and named for the particular device, build type and date. Each time you run a build script a new directory will be created to hold the firmware files produced by that build run.
Customising the Setup Scripts
The Setup scripts by default use specific versions of the OpenWrt codebase and packeges. Because the version of the OpenWrt codebase and the packages are locked to specific versions, anyone setting up a build environment with the script will be able to generate the same firmware.
If different versions of the OpenWrt codebase or the packages are required, then these can be selected as required by editing the script asdescribed below.
Setting the OpenWrt Code Revision
The OpenWrt codebase has branches corresponding to major releases such as Barrier Breaker (14.07) and Chaos Calmer (15.05)
The details of these branches are available here:
Within a branch, there will be updates from time to time.
The revision number shown against the branch on the above web page represents the latest update to the branch (eg chaos calmer r48666).
The Setup scripts allow you to either take the latest updated version of the branch, or to specify a particular update version by specifying the SHA value for the revision in Git.
To see all the revisions of a particular branch, you can check out the Git repository for the branch and browse through it with a suitable Git browser such as Gitg.
To get a copy of the repositories for BB and CC releases, use the following commands:
$ cd ~/Git $ git clone git://git.openwrt.org/14.07/openwrt.git ./openwrt-14.07 $ git clone git://git.openwrt.org/15.05/openwrt.git ./openwrt-15.05
To get a copy of the repository for trunk, use the following command:
$ git clone git://git.openwrt.org/openwrt.git ./openwrt-trunk
This will create directories ~/Git/openwrt-14.07 , ~/Git/openwrt-15.05 and ~/Git/openwrt-trunk containing the repositories which can be browsed with a tool such as Gitg.
To update the repository at any time use the commands:
$ cd ~/Git/openwrt-14.07 or $ cd ~/Git/openwrt-15.05 or $ cd ~/Git/openwrt-trunk $ git pull origin
The Setup script contains a code block at the top of the script similar to the following:
# Set default version parameters REVISION="15.05" SETSHA="TRUE" SHA="48e7befb" # 15.05 branch as at 5 Aug 2015
If SETSHA is set to FALSE, then the latest revision of the branch will be used.
If SETSHA is set to TRUE, then the revision corresponding to the specified SHA value will be used.
By default, the Setup scripts have SETSHA set to TRUE and so will use a revision with an explicitly specified SHA value, generally corresponding to the revision used to build the current stable version of the SECN code.
Setting the version of Packages
Similarly, by default, the script will lock the feeds for various packages to specific versions in the "feeds.conf.default" file.
You can edit this file to change the version of one or more package feeds if required. The original file is saved as "feed.conf.default.bak"
To update the feeds after making a change, use the following commands:
$ cd ~/openwrt/Build-14.07 $ ./scripts/feeds update -a
Setting up a Build Environment Manually
This section of the document assumes that you are setting up a build environment on Ubuntu 14.04 LTS and using the "Barrier Breaker" revision of OpenWrt.
References: OpenWrt Wiki » Documentation » HOWTOs » OpenWrt Buildroot – Installation
- A PC with Ubuntu operating system installed. The process below has been tested on Ubuntu 14.04 LTS - 500 MB of hard disk space for the source files to be downloaded. - 5 GB of available hard disk space to build (i.e. cross-compile) OpenWrt.
- Do everything as non-root user! Use your normal login account to issue commands, using sudo where indicated. - Issue all OpenWrt Buildroot commands in the <buildsystem root> directory e.g. ~/openwrt/attitude/ - Do not try to build in a directory that has spaces in its full path.
Install Git and Subversion
Git and Subversion (svn) allow you to conveniently download the OpenWrt source code and build tools to assist with the compilation process.
Install Git, Subversion and the build-essential tools as follows:
sudo apt-get install git-core subversion build-essential gawk zlib1g-dev python
For information about Git see git 
Download the OpenWrt sources
For general information about OpenWrt code versions see:
https://openwrt.org/ https://openwrt.org/releases/15.05/start https://openwrt.org/releases/17.01/start
To see recent updates to various branches of the source code, check these sites:
Click on a branch name to expand and see details of changes, for example:
Note: SECN firmware on the VT Downloads site is generally built .with the latest revision for the particular branch (eg AA, BB) as shown in the above pages at the time of the build.
The required source code can be checked out of the Subversion repository and downloaded as follows:
cd mkdir ~/openwrt cd openwrt For the original Attitude Adjustment 12.09 stable release (r36420): svn checkout --revision=36420 svn://svn.openwrt.org/openwrt/branches/attitude_adjustment/ my-dir For an updated Attitude Adjustment 12.09 release (for example r42647): svn checkout --revision=r42647 svn://svn.openwrt.org/openwrt/branches/attitude_adjustment/ my-dir For an updated Barrier Breaker release (r43321): svn checkout --revision=r43321 svn://svn.openwrt.org/openwrt/branches/barrier_breaker/ Or for a particular trunk (development) revision e.g. rev 36420 svn checkout --revision=36420 svn://svn.openwrt.org/openwrt/trunk my-dir
This will create a directory 'openwrt' in your home directory, then check out the required revision into a new sub-directory:
which will contain the OpenWrt source code, and will include the OpenWrt Buildroot system.
Note: To view the contents of the SVN repository use the command:
svn ls svn://svn.openwrt.org/openwrt/branches/ svn info svn://svn.openwrt.org/openwrt/branches/ svn ls svn://svn.openwrt.org/openwrt/branches/attitude_adjustment svn info svn://svn.openwrt.org/openwrt/branches/attitude_adjustment svn ls svn://svn.openwrt.org/openwrt/branches/barrier_breaker svn info svn://svn.openwrt.org/openwrt/branches/barrier_breaker
Note: OpenWrt also provide pre-built firmware for all targets. To see the firmware available for download, browse to:
To see all available downloads, including trunk snapshots: http://downloads.openwrt.org/ For the original Attitude Adjustment 12.09 stable release: http://downloads.openwrt.org/attitude_adjustment/12.09 For the original Barrier Breaker 14.07 stable release: http://downloads.openwrt.org/barrier_breaker/14.07/
Set up Package Feeds
Backup the file 'feeds.conf.default' in the build directory.
cp ./feeds.conf.default ./feeds.conf.default.orig
Open the feeds.conf.default file for editing and edit the line that checks out the package feed for the specified OpenWrt revision.
This is usually the first line in the file, and may already be set to the correct value, depending on how you checked out the code.
Edit the line as follows:
For Attitude Adjustment 12.09 stable release: src-svn packages svn://svn.openwrt.org/openwrt/branches/p[email protected] For the updated Attitude Adjustment 12.09 release: src-svn packages svn://svn.openwrt.org/openwrt/branches/[email protected] Or for a particular trunk revision e.g. rev 36420 src-svn packages svn://svn.openwrt.org/openwrt/[email protected]
From Attitude Adjustment 12.09 RC2 onwards, batman-adv is no longer included by default in the OpenWrt build, so the feed has to be added if you are building a version of OpenWrt later than AA RC2.
Similarly for Telephony / Asterisk in Barrier Breaker.
Add lines for the feed for batman-adv, telephony and for ALFRED:
For Attitude Adjustment 12.09 Stable release: src-git routing git://github.com/openwrt-routing/packages.git;for-12.09.x For Barrier Breaker RC release: src-git telephony http://git.openwrt.org/feed/telephony.git src-git routing https://github.com/openwrt-routing/packages.git;for-14.07 For trunk release: src-git routing git://github.com/openwrt-routing/packages.git And for all: src-git alfred git://git.open-mesh.org/openwrt-feed-alfred.git
You may also comment out any unwanted feeds that are active e.g. x-wrt, luci
Download feeds using the 'feeds' script:
./scripts/feeds update -a
Install additional packages
Install additional packages including batman-adv and openssh-sftp-server:
./scripts/feeds install kmod-batman-adv ./scripts/feeds install openssh-sftp-server ./scripts/feeds install usb-modeswitch ./scripts/feeds install usb-modeswitch-data ./scripts/feeds install kmod-usb-serial ./scripts/feeds install kmod-usb-serial-option ./scripts/feeds install kmod-usb-serial-wwan ./scripts/feeds install haserl ./scripts/feeds install xinetd ./scripts/feeds install muninlite ...
Alternatively install all packages with:
./scripts/feeds install -a
Check for missing Ubuntu system packages
Check for missing Ubuntu packages on the system on which you want to build OpenWrt:
This will list missing Ubuntu operating system packages needed to successfully build OpenWrt using buildroot. Install any missing packages using the normal package management commands (e.g. apt-get install <package_name>), for example on Ubuntu 14.04:
sudo apt-get update sudo apt-get install zlib1g-dev sudo apt-get install libncurses5-dev sudo apt-get install gawk sudo apt-get install git-core sudo apt-get install gettext sudo apt-get install libssl-dev
Set up the Device Configuration
Run the 'menuconfig' graphical tool to set up the details of the required build:
A guide to commands that can be used in the tool are shown at the top of the screen.
For basic navigation around the page, use the arrow keys, up and down to highlight items, left and right for Select and Exit to move between pages.
To select an item for inclusion in the build, enter "Y", or "N" to deselect.
Select the required device family and type in the menu.
For MP-01, Ubiquity Pico, Nano etc based on AR231x, or MP-01 select:
Target System 'Atheros AR231x/AR5312' Target Profile 'Default'
For TP-Link WR703N, MR11U, WR842ND, MR3020, WDR4300 etc, select:
Target System 'Atheros AR71xxx/AR...' Subtarget 'Generic' Target Profile 'TP-Link TL-WR703N' Select required device type.
For Ubiquity Pico-M etc select 'Atheros AR71xxx/AR...' select:
Target System 'Atheros AR71xxx/AR...' Subtarget 'Generic' Target Profile 'Ubiquity Products'
NOTE: Using the AR231x image from this build environment for MP-01 will not support the telephony subsystem.
A special build environment is required to support the build of the device drivers etc.
A separate wiki article will cover that.
Select Target Image Formats
The build process will produce a variety of firmware file formats. The most commonly used format is 'squashfs' and this is produced by default.
By default, the 'jffs' and 'tar.gz' formats will also be selected and you can de-select these options if you do not require these formats.
Select the required packages for the VT SECN build located in the menu as indicated:
wireless-tools Base System openssh-sftp-server Network > SSH See Note 1 uhttpd Network > Web servers uhttpd-mod-tls Network > Web servers See Note 1 re SSL support xinetd Network kmod-fs-ext4 Kernel modules > File Systems kmod-fs-vfat Kernel modules > File Systems kmod-nls-cp437 Kernel modules > Native Language Support kmod-nls-iso8859-1 Kernel modules > Native Language Support kmod-batman-adv Kernel modules > Network Support kmod-usb-ohci Kernel modules > USB Support kmod-usb-serial Kernel modules > USB Support kmod-usb-storage Kernel modules > USB Support muninlite Administration comgt Utilities haserl Utilities iwinfo Utilities pxg5 Utilities usb-modeswitch Utilities ... Note 1: The use of sftp-server requires selection of 'openssl' support for uhttpd-tls, but uses an additional 100kB of flash memory. On devices with 4MB Flash, do not select openssh-sftp-server and leave the selection of SSL support for uhttpd-tls as the default 'cyassl' to ensure firmware size does not exceed limit.
When exiting the 'menuconfig' graphical tool, select 'Save Config' to save the configuration you have created.
The configuration will be stored in the file:
'~/openwrt/attitude/.config' OR '~/openwrt/rev-34386/.config'
This file can be copied and stored for later reuse.
NOTE: If you copy a saved '.config' file back into the build directory later, it is necessary to either run the 'menuconfig' tool again or run the 'defconfig' command, which is equivalent to opening the 'menuconfig' tool and closing it again without making any changes.
NOTE: The package 'uhttpd-mod-tls' defaults to using the 'cyassl' library. If a make process is run with this selection, and subsequently the selection is changed to 'openssl', it will be necessary to do a 'make clean' of the uhttpd package to avoid a build time error.
Also, in this scenario, the 'cyassl' library will still be loaded, taking up quite a bit of flash memory space which is critical on small devices. You have to look in the 'Libraries' section of the 'menuconfig' menu and manually de-select the cyassl library.
Add Custom Files
By default, the 'make' process will build a firmware image with the default OpenWrt configuration for the selected device. To add your own files to customise the firmware, create a './files' directory and build the directory structure you want to overlay on the base files in the build e.g
./files/www/cgi-bin/my-cgi-script.sh ./files/etc/config/network ./files/etc/config/batman-adv ./files/etc/init.d/my-init-file.sh
These files will be installed on the device, replacing any default files of the same name. In this way the configuration and operation of the device can be customised.
For VT firmware, the 'files' directory and '.config' file required to build the standard firmware images are generally published in the VillageTelco GitHub repository.
Proceed with the Build
Run 'make' to build the firmware images. The process will take longer to run the first time as the sources are compiled. It may take 20 minutes or more on a typical Laptop PC. Subsequent builds will be faster as only changes will be re-compiled.
Collecting the output firmware files
The output files will be placed in the directory
depending on the device family selected.
Operating System Prequisites
Additional pre-reqs may be required depending on the Ubuntu version you are using. Some examples follow:
Ubuntu: $ sudo apt-get install build-essential subversion libncurses5-dev zlib1g-dev gawk flex
Ubuntu 9.10 $ sudo apt-get install gcc-multilib bison autoconf screen gcc g++ binutils patch bzip2 flex make gettext unzip libc6 git-core
Ubuntu 11.10 $ sudo apt-get install build-essential subversion git-core libncurses5-dev zlib1g-dev gawk flex quilt
Ubuntu 64bit $ sudo apt-get install build-essential subversion libncurses5-dev zlib1g-dev gawk gcc-multilib flex git-core gettext
Using Release Sources (stable)
To check out the source code that the stable 'backfire' or 'attitude_adjustment' release is built from (plus the latest backported fixes from trunk):
svn checkout svn://svn.openwrt.org/openwrt/branches/backfire OR svn checkout svn://svn.openwrt.org/openwrt/branches/attitude_adjustment
To check out a 'tagged' version of the 'backfire' sources (ie no backports):
svn checkout svn://svn.openwrt.org/openwrt/tags/backfire_10.03 OR svn checkout svn://svn.openwrt.org/openwrt/tags/attitude_adjustment_12.09/
To see what 'tagged' versions are available:
Using Development Sources (bleeding edge)
The development branch (trunk) contains everything from documentation to experimental patches.
svn checkout svn://svn.openwrt.org/openwrt/trunk/
Beware of unusual environment variables such as GREP_OPTIONS which should not have '-initial-tab' or other options affecting its output
SED should not be set. If it is, run `unset SED` before compiling.