Creating an OpenWrt package for a web page

From PRPL
Jump to: navigation, search

Based upon the then-latest development version of OpenWrt, Chaos Calmer

Like most distributions of Linux, OpenWrt has a rich package management system. One particular advantage of OpenWrt is how simple and straightforward it is to create these packages. In this tutorial, you’ll learn how to:

  • build a new package for hosting a single web page
  • create and install a feed to host your package
  • include your package in an OpenWrt build
  • boot your new copy of OpenWrt in QEMU and
  • test that your web page is accessible on the host computer

Before you begin[edit]

Like in the last tutorial, you're going to need to be using Linux, Mac or other Unix operating system. Unfortunately, Windows is not supported for building OpenWrt but you can run a Linux distribution in a virtual machine on Windows.

For this tutorial, I'll use the commands necessary for building on Ubuntu Linux 14.10 - 64-bit (Ubuntu 14.04 would likely always work but it hasn't been tested). The instructions should work for Debian, however Fedora and MacOS will be different and not be covered in this tutorial.

To get started, make sure you have a terminal window open and have installed the prerequisites for building OpenWrt. If you completed the previous tutorial, you should know how to open a terminal window and have installed the prerequisites! If you haven't completed the previous tutorial (or just want to make sure everything is set up properly), follow the directions in these sections from the previous tutorial:

  1. Open a terminal window
  2. Installing prerequisites

Getting the latest OpenWrt code[edit]

If you already have the OpenWrt code[edit]

To be safe, we're just going to undo all our changes to the OpenWrt code base and get us back to a known state. Assuming you haven't committed to the git repository, which you would not have in the previous tutorial, we'll run:

git reset HEAD --hard

This resets the commit of the working tree to what it was on the last commit. The --hard also gets rid of any changes that would have been made in the working tree so be sure you don't have anything you want to keep!

Next, let's grab the latest version of the OpenWrt code by running the following command:

git pull --rebase origin

This updates your local copy of the OpenWrt code to latest official revision.

If you want to use the exact revision of the OpenWrt code base that I used, run the following command:

git checkout 6f9f3e0bf01a50ca7d32934d84968cd06ba07ea0

This makes sure you'd building with the exact same version of the code base as I am.

Let's go back to our parent directory to get started:

cd ..

If you're getting it for the first time[edit]

The OpenWrt source code is stored in a Git repository. Getting OpenWrt consists of a single command:

git clone git://git.openwrt.org/openwrt.git

This gets you a copy of the OpenWrt code base on your local system.

If you want to use the exact revision of the OpenWrt code base that I used, run the following commands:

cd openwrt

git checkout 6f9f3e0bf01a50ca7d32934d84968cd06ba07ea0

This makes sure you'd building with the exact same version of the code base as I am.

Let's go back to our parent directory to get started:

cd ..

Build a new package[edit]

Now that our copy of OpenWrt is ready, let's actually dive into creating our package.

Creating the directory structure[edit]

To get started, we need to create a directory structure for our package to sit. It's not going to be inside our copy of OpenWrt but in a slightly different location. Right now, it may be unclear why this is but as we go through the tutorial, you'll learn why.

First, let's create a directory and change our directory to it:

mkdir ourfeed

cd ourfeed

Why is the directory called ourfeed? We'll find out soon!

Next, we're going to create a directory for our package. Since we're creating a package to install a webpage on our OpenWrt build to help us learn, let's call it learning_webpage. We'll create the directory and then change our directory to it:

mkdir learning_webpage

cd learning_webpage

Writing our Makefile[edit]

Every OpenWrt package has a Makefile. Makefile's are traditionally used for compiling code. In the case of OpenWrt packages though, Makefiles serve additional purposes. A package Makefile contains instructions for installation, where the package should be placed in the OpenWrt build menu, its description and more. These package Makefiles are almost like description files but they do have the traditional capabilities of a Makefile.

For writing our Makefile our instructions will use nano as our editor. That said, you can use any editor you're comfortable with.

To get started, we'll start editing our Makefile by running:

nano Makefile

Next we're going to paste the following code into our Makefile. To paste, move the text cursor in nano to the appropriate place, drag the mouse cursor into the terminal window, right click and select Paste:

Now you're probably wondering what all this means but have no fear! We're going to go through our Makefile and help you understand what everything does.

Standard includes[edit]

OpenWrt requires some standard includes for Makefiles. Fortunately, these standard includes contain some really cool and useful tools. On Line 1, we have the standard include for almost every Makefile in OpenWrt. The rule.mk file has lots of macro definitions for condensing commands and keeping consistency throughout OpenWrt. Later on we'll have another include as well but we'll wait until we get there to discuss it.

Basic package info[edit]

Every package definition requires some basic information like name, version and some other info. In the next several lines, we'll be setting that information. On Line 3 we provide the name of our package as shown in the build menu screen and the ipkg filename. If you make your own, make sure you don't use spaces.

Line 4 has the version number of our package. We're being very lazy here and setting it to 1 here. In other cases you may want to get this number from another file or maybe from the software you're packaging. Remember, this is a Makefile so you can do anything here that you could do in any Makefile.

Next we set the release version of our Makefile on Line 5. There are lots of situations where you might want to modify your Makefile but not actually increment the package version number. For example, you may have made a mistake in your description but your underlying software hasn't been changed; release version of your Makefile is where you change that. In our case we set it to $(PKG_SOURCE_VERSION) which defaults to current date.

On Line 6 we have you might think is the most important bit of information: the package maintainer. Why is it so important? Because that's you! Make sure to add your name and email in the format shown to get the recognition you deserve.

Every package has one or more software licenses which the software it contains is provided to the user under. In our case, we set our software license on Line 7. OpenWrt uses the SPDX license identifiers to help package users better understand which license a package is licensed under. In our case we're licensing our package under the ISC license which uses the identifier of ISC. If you ever have a package licensed under multiple license, you should add all the license identifiers to this field with a space in-between each.

package.mk include[edit]

On Line 9 we have our last Makefile include, our package.mk file. Every package Makefile should have this include which pulls in all the rules needed to actually create OpenWrt package files.

Package/learning_webpage/default directive[edit]

As mentioned before, our package Makefile is just a normal Makefile being used in a unique way. Because it's an normal Makefile, we can use define directives. Define directives can be separated into two types in OpenWrt. One type, which we'll call "standard directives", are directives that OpenWrt expects to exist as part of a standard build of a package. In this sense, they're a bit like what one might call an interface in Java or C++. The other type, which we'll call "custom directives", are directives which a Makefile writer might use for code reuse, just like in standard Makefile.

So what kind of directive is the Package/learning_website/default directive? It's a custom directive. We're using this directive for declaring information which will be used as part of the OpenWrt build menu. OpenWrt has a menu system for configuration a build which which we'll use later and it's important that our package work properly in that menu.

On Line 12, we set the top level submenu for our package. Since we're creating a website, we'll pick the top-level Network submenu for our package. We can select an even more specific category than just Network for our menu. In Network there's a submenu titled Web Servers/Proxies; that might fit us better. We set that submenu under Network at Line 13.

learning_webpage is a perfectly fine title for your package but it's not very descriptive. Fortunately, we can add a short description here which will show up in the build menu so others know what our package actually does. We're setting that at Line 14. Remember, it needs to easily fit on less than a single console line.

Package/learning_webpage directive[edit]

The next rule is a standard directive. In this case, this rule is for setting up information standard configuration information for packaging and building your package. As you may have noticed, the OpenWrt has a standard mechanism for naming Package define directives: Package/package_name(/optional_additional_subdirectivename). This really assists in simplifying reading and maintaining package Makefiles for packages which you didn't create.

Inside our Package/learning_webpage directive, on Line 18, we include the contents of Package/learning_webpage/default directive. Like any Makefile, it's possible to include the contents of another definition in a different place.

Every OpenWrt package can require that other packages be installed prior to its own installation. This is an example of the rich dependency management that OpenWrt supports for packages. In our case, we need to require a web server, uHTTPd, be installed before installing our own package. After all, since we're going to installing a web page with this package, we need a web server to actually serve that page! We declare that dependency on Line 19. Package dependencies are described with a special format but in our case the string is pretty simple: +uhttpd: uhttpd is the package title of the uHttpd package and the + ahead of it means our package requires it be installed. If you have multiple dependencies for your package, you can add more of them by adding more of additional identifiers on this line separated by spaces.

Package/learning_webpage/description directive[edit]

Another standard OpenWrt directive, our Package/learning_webpage/description section contains a free text description of the package. This description is used in the build menu if a user wants to learn more about a package beyond our short description earlier. Our additional, longer description is at Line 23.

Package/learning_webpage/install directive[edit]

We've mostly described our package so far but we haven't placed any files into it yet. We will fix that here! Another standard OpenWrt directive, Package/learning_webpage/install contains the commands for copying files from our source package directory into the compiled package file. On installation the contents of the package file will be copied to the device image.

We're going to actually perform a copy of all our files on Line 27 from our local package directory into the proper location in our package file. Some items of note:

  • $(CP): We use $(CP) instead of cp because the $(CP) macro sets a number of options for making recursive copies and installs more reliable (-fpR). This macro is declared in the rules.mk file we included on the first line.
  • $(1): The $(1) variable corresponds to the root directory of you package file's directory hierarchy.

Build/Compile directive[edit]

Many packages compile software as part of their creation so the Build/Compile directive defaults to running make. In our case, we don't have anything to compile so we need to override it on Line 30 by simply adding the word true into our directive. If we didn't override this directive, our package would fail to build.

Final build package call[edit]

Every package Makefile must end with a call to BuildPackage with the name of the package being built. In our case, this happens at Line 34. If you changed the name of our package, you'd replace "learning_webpage" on this line with the new name of your package.

Saving our Makefile[edit]

Now that we've worked our way through our Makefile, we need to close and save it by holding Ctrl and pressing the x key. Follow the prompts to save our file as Makefile and you'll be taken back to the the command line.

Create our package contents[edit]

We've set up our Makefile but we haven't created our actual contents. If you remember from our Makefile, we're going to copy everything inside our files subdirectory into our package. We don't have a files subdirectory so we better create it and change to this directory:

mkdir files

cd files

Create and write our web page[edit]

We're in our files directory and we're going to have to create the first of two subdirectories in files: the www directory. We create this directory and switch to it using these commands:

mkdir www

cd www

We're in our www directory which is where we're going to put our web page. In our case, our webpage will consist of a single, very simple index.html file. We'll start editing our web page by running:

nano index.html

Add our new feed[edit]

Next, we'll paste in the following code:

This is a very basic html file and discussing what it does is beyond the scope of this tutorial. We don't have anything to discuss or edit so let's just save this file as index.html and close nano.

Let's go back up to our files directory by running:

cd ..

Write our first time boot script[edit]

Some packages requires additional configuration once running on the device. For example, packages might need to setup being run as a service or open ports in the firewall. In our case we want to change the IP address of the device. Packages can makes these changes in a first time boot script. We're going to go ahead and create that now.

First we need to create the etc and inside it the uci-defaults directory. On an OpenWrt device, the /etc/uci-defaults directory contains first time boot scripts which run on the first boot and are removed upon finishing successfully. Let's create our directory structure and change to the uci-defaults directory now:

mkdir -p "etc/uci-defaults"

cd etc/uci-defaults

Now that we're in the proper folder, we're going to create our boot script. Before we do that, we should talk about the name of this file.

Our script file will be named 99_set-network-ipaddr. Why do we use the name? The scripts in uci-default are run at boot time in order based upon the leading digits of the filename, from 01 to 99. We want our file to run after everything else has run so we start the name with 99. We then add an underscore to separate the number from the rest of the file name and then use set-network-ipaddr as a simple description of what the script actually does.

Let's actually start editing our file by running:

nano 99_set-network-ipaddr

Next, we'll paste the following code into nano:

Let's go through this script. On Line 1 we have the standard shebang line needed for all shell scripts. Line 3 though is where we get to see a cool feature of OpenWrt: UCI.

UCI, short for Unified Configuration Interface, is a command line tool available on an OpenWrt device for managing configuration files. UCI provides a common way of getting and setting configuration for many programs at the command line. Additionally, UCI has support for staging changes until you "commit" them to the configuration location.

Let's go through each part of Line 3 to make sure we understand what's going on. uci is the name of the UCI program of course. -q means to not print error messages; since this is running automatically at boot, no one would be able to see them anyway. set is the command for saying we're going to set a configuration value. network.lan.ipaddr=10.0.2.15 sets our IP address to 10.0.2.15 which is the default address QEMU provides to our VM for the networking setup we'll be using.

On the next line, we run another UCI command but in this case, we run commit. This command says we'll save our IP address change - remember, UCI stages changes. To specify we want to save any changes to our network settings, we specify network at the end.

The last line in our script returns a 0. Any UCI-default bootscript which doesn't return a 0 will be run again at the next boot. If there was a serious error, you might return an error but we won't handle that here.

Now that we've worked our way through our boot script, we need to save our file as 99_set-network-ipaddr and close nano to get back to the command line.

Adding our feed[edit]

Now that we've created our package you might be wondering why we created our package in a location that's not under our openwrt directory. The answer is actually in the name of one of the directories we created, particularly our_feed. our_feed will be a new 'package feed' we add to OpenWrt. A package feeds is a listing of pieces of software that you can install in an OpenWrt instance. OpenWrt comes with a bunch of package feeds which includes dozens of pieces of software. One of the great things about OpenWrt package feeds is how simple they are. In fact, a directory with subdirectories containing packages is enough to make a feed! Feeds can be accessed via Git, SVN and number of other mechanisms but in our case, our feed will be access via a symlink.

To get started on this we're going to go back to our OpenWrt directory:

cd ../../../../../openwrt

Next, we need to set up our feeds.conf file. feeds.conf is the list of all the package feeds OpenWrt can get software from. OpenWrt has a default feeds file, feeds.conf.default, as a backup in case we didn't have a feeds.conf file. We could modify that but, just to be safe, we're going to make our own feeds.conf file. We'll start by creating it from feeds.conf.default:

cp feeds.conf.default feeds.conf

We have our feeds list but we still haven't added our own feed with our cool web page to the list yet. Let's do that now. To get started, we need to get the absolute path to our feed directory. To do that, we run:

realpath ../ourfeed

What realpath returned was the absolute path to ourfeed. In my case, it was /home/eric/ourfeed. Either copy or write down that feed path, we're going to need it in a second.

Next, let's start editing feeds.conf in nano:

nano feeds.conf

The file that shows up should look like this:

The feeds.conf file has a simple format. Using the first line as an example, it consists of a method to retrieve the feed (src-git), a short name for the feed (packages) and path to the feed (https://github.com/openwrt/packages.git).

To add our feed, we're going to replace the last line with the following:

src-link custom /your/path/to/ourfeed

Make sure to change /your/path/to/ourfeed to the path returned from realpath before. In my case, that was /home/eric/ourfeed Make sure your line doesn't start with a pound sign (#) If your line starts with a pound, OpenWrt will ignore that feed!

We're done editing our feed now so we need to close nano and save it our file as feeds.conf and you'll be taken back to the the command line.

Updating our feed and preparing our package for build[edit]

Now that we've edited our feed list, OpenWrt needs to read the list of feeds and get a list of packages from those feeds. We do that by running:

scripts/feeds update

We need to have our package included in the build menu. To do that we run:

scripts/feeds install learning_webpage

It's important to note that, despite the word install in this command, we're not installing our webpage onto our OpenWrt image quite yet. Think of this command as making our package available for an OpenWrt image build if we choose it to be installed.

Preparing our image[edit]

The Linux kernel and the set of software that runs on a router or, in our case, QEMU gets combined into a file called an image. If you're familiar with the concept of an ISO file, an image is a little bit like that. We need to set up what software and features are included in our image. Fortunately OpenWrt has a graphical tool for this called menuconfig.

To get started run we need to start the menuconfig tool in our terminal by running:

make menuconfig

After a few seconds, the following screen should pop up in your terminal window:

Buildmenu.png

This is the main configuration screen for setting up your OpenWrt image. OpenWrt includes options for thousands of different system configurations; we won't address many of them in this tutorial but suffice it to say, there's lots of opportunities to do creative things if you want to.

Setting Target System[edit]

The first thing we need to do is to set our target system. The target system option corresponds to a processor type or family. In most cases, routers will not function if they have an image from an incorrect target system type. To do that, we press Enter to go to the target system submenu.

Atheros selected.png

By default, the target system is Atheros AR7xxx/AR9xxx. We want to use the "MIPS Malta CoreLV board (qemu)" instead. Simply press your down arrow key repeatedly until you're highlighting "MIPS Malta CoreLV board (qemu)"

MIPS selected.png

Press Space to select this. You should be back at the main configuration screen now with the proper target system selected.

Setting Advanced Configuration Options[edit]

Occasionally, developers need to be able to modify the build process of OpenWrt slightly. OpenWrt places these options in the "Advanced configuration options (for developers)" menu. Due to a bug in the version of QEMU included with Ubuntu 14.10, we need to modify ones of these options. If you're not using Ubuntu 14.10, it's possible you won't need to perform this step.

To get started, we use the down arrow key to highlight the Advanced configuration options (for developers) menu item. Then we press Space to tell OpenWrt that we'll be using one of these options.

Advanced Config selected.png

Next, we press Enter which takes us to the Advanced configuration options sub menu.

There are loads of choices in this menu for modifying all kinds of things of settings affecting the OpenWrt build process. In our case we're going to need to modify one of the "Target Options" settings. To do that, we use the down arrow key to highlight the Target Options menu item. Next we press Space to tell OpenWrt that we want to modify one of the Target Options.

Build package target options selected.png

To go to the Target Options submenu, we press Enter. There will be a few options that might not mean much to you right now; that's okay. We need to turn the "Build packages with MIPS16 instructions" off. We'll do that by using the down arrow key to highlight the "Build packages with MIPS16 instructions" menu item and then pressing Space to deselect this option.

MIPS16 turned off.png

Now that we've changes this option, we'll press the right arrow once to highlight Exit and then Enter twice to get back to our main menu.

Selecting our custom package[edit]

We've selected all the architecture and image creation settings we're going to need. What we haven't selected is our custom package. After all, the point of all this was to add web page package to an OpenWrt image.

We set our package as being in the Network menu and the Web Servers/Proxies submenu so we need to go there to select it. We can get started by pressing the down arrow until we get to the Network menu item.

Network selected.png

We press the left arrow once to high light Select and then Enter which takes us to the Network category menu. Here is a list of packages and categories of packages related to networking. We need to go to Web Servers/Proxies menu item by pressing the down arrow.

Web servers selected.png

We press Enter and whoa, there's our package!

Learning webpage not selected.png

The first thing you should notice is that learning_webpage is not selected nor is uhttpd. We're going to select those in just a second.

You probably see your package name and your short description from before but where is the description? Well that's available via the package help. To get there type a ?. You'll see more details about your package. To get back to last screen just press Enter

So now that we're back at the package selection screen, we want to actually select learning_webpage. To do that we type y. Your screen should look like this now:

Learning webpage selected.png

Notice that not only was learning_webpage selected but uhttpd was also. Why was that? Because remember, we set uhttpd as a required dependency of our learning_webpage package. In fact, if you tried to deselect uhttpd now , the build menu would not allow you to unless you deselected learning_webpage first.

Saving our .config[edit]

We're done setting up our image so now we just need to save our image. We do that by exiting the build menu screen. Press right to select the Exit "button" at the bottom and press Enter. Do that a few more times until it asks you if you'd like to save your configuration. We want to so press Enter. Our configuration is saved and we should be back at the terminal.

Compile the image[edit]

Compiling the image is pretty simple, all you have to do is run:

make

This command will take a pretty long time to finish, possibly hours. Don't worry; that's normal!

Boot our image in QEMU[edit]

Once our the compilation is complete, it's time to actually boot our image in QEMU! To do that, we run the following command:

qemu-system-mipsel -kernel bin/malta/openwrt-malta-le-vmlinux-initramfs.elf -nographic -redir tcp:5555::80 -net nic -net user

Let's break down each part of this command:

  • qemu-system-mipsel is the QEMU command to start a new emulated MIPS system in little endian mode.
  • -kernel bin/malta/openwrt-malta-le-vmlinux-initramfs.elf is an option telling QEMU that our kernel image for our operating system is in the bin/malta/openwrt-malta-le-vmlinux-initramfs.elf image.
  • -nographic is an option telling QEMU to turn off the normal graphical output and send all of the command line input and output to the terminal.
  • -redir tcp:5555::80 tells QEMU to redirect TCP calls from the port 5555 on your computer to port 80 in OpenWrt image.
  • -net nic tells QEMU to add a virtual network card in our OpenWrt virtual machine
  • -net user tells QEMU to connect our virtual network card to our host computer in user-mode.

You should see lines of text show up on your screen. Everything that's happening is the normal boot mechanism for OpenWrt.

View our webpage[edit]

I know you're excited to view the webpage but we have to wait for the network stack on OpenWrt to be ready. Fortunately, this won't take long. We know the network stack and web server is ready for serving content when you see the following line of text in the terminal:

br-lan: port 1(eth0) entered forwarding state

Once you do, open your favorite web browser on the host and go to http://localhost:5555. The page your browser loaded should look like this:

Learning webpage display.png

We're serving a custom web page installed in our very own package onto our own build of OpenWrt. That's pretty cool, isn't it?

Shutting down QEMU[edit]

When you're done playing around in QEMU and marvelling at your web page, you're going to want to turn it off just like you would a real machine. We can do this in QEMU by going back to our terminal window and pressing Ctrl-a followed by c. This gets us to a console for controlling QEMU virtual machines. From here, we run the command q. This sends us back to our main command line and and shuts down our virtual machine.

Wrap up[edit]

Through this tutorial, we've explored how to create a package, select it for installation into an image, build that image and test that it's working in QEMU. As you explore OpenWrt packages, it's important to realize that they have multiple uses. Because of the nature of the dependency model, they can not only be used for packaging software but also for creating a flavor for an entire router image.