Diary of a Port – The Dillo Web Browser

So, you want to do a port for the Pandora? But you have no idea where to begin with ? You are not familiar with Linux either ? You don’t feel like going through thousands of forum pages, with people contradicting each other every 2 posts, to find out the right way to do it was on page 56 out of 81 ? Well aren’t you the lucky one, today Dredd shows you how to do a port from start to finish.

For this post, I built the Dillo browser, it’s fast and light with no Javascript, now on the repo.

Here are my notes on setting up the Pandora for development, and how I did this Dillo port.  If you like, you can follow along and build Dillo again, or have a go with another app … maybe start with a simple shell script, like:  zenity --info --text 'hello, world'

Please do comment here… if you get stuck with a port, if you see a mistake, if you know a better way, if you give up in horror … and especially if you do succeed to port anything!

Overview

We’ll start with some basic setup:

• Connect to the Pandora with SSH
• Prepare your SD cards
• Make folders and symlinks
• Install dev tools

Then move on to the port proper:

• Start code::blocks cli
• Get source code for the app
• Fetch and build needed libs
• Try to build the app
• Install the app
• Make special changes for Pandora
• An interlude: more than you ever wanted to know about the PND system
• Prepare PXML, icon, run script, screenshots
• Build the pnd
• Test and fix the pnd
• Distribute your port

The setup will take maybe 30 minutes, plus a big download.  Porting Dillo took me a few hours, it was an easy port.  If you follow this guide, maybe you could do it in one hour.  Some apps depend on many libs, or need major work done, it can be much more difficult than this.

Connect to the Pandora with SSH

• We use ssh (secure shell) to connect to Pandora from a PC, run commands, edit files.

• You’ll go mad, if you do all this typing on the Pandora keyboard.  _wb_ is special!
• Even if you love typing on the Pandora, ssh is useful for copying files, fixing X, etc.
• Set good strong passwords for your user account and for root.  Don’t forget them!

  passwd         # set a password for your user account
  sudo passwd    # set a password for the root account

• Pandora -> Settings -> Startup: Enable services, check “Start Dropbear“, OK and restart.
• Make Pandora use the same IP address each time, and put it in /etc/hosts on your PC.
• SSH connect from your PC, like  ssh sam@floss , i.e.  ssh user@pandora-name, or use putty.
• You can also use sshfs, scp, or rsync to copy files. Check google or their manpages.
• Run this qos “quality of service” script, it makes Pandora WIFI better (e.g. for ssh).

  sudo iwconfig wlan0 power off rate 54M

Prepare your SD cards

• You need good, large >= class 10 SD cards. Don’t buy slow cards. Don’t buy on e-bay!
• I use ext4 on my main dev SD card – ext4 can do symlinks and hard links. Yay!
• Put FAT on another SD card for testing, it can be a small one
• NTFS can be good if you need windows compatibility, may be some performance issues
• Here’s how to format your SD cards, you can do it with the Pandora or a PC
• I suggest use nice short labels, mine are called “1p” and “2p”.

Make folders and symlinks

• This is my personal setup, you can do things similar or differently
• Folders for projects and scripts:

  cd /media/1p                 # my first SD card is called "1p", nice
  mkdir code                   # a folder for code
  mkdir code/x                 # for scripts
  mkdir install                # for installing apps before making a PND
  ln -s $PWD/code/x ~/bin      # link the scripts folder to ~/bin

• Symlinks useful folders to the root – save your fingers from too much typing!

  sudo ln -s /media/1p /1p               # for the 1st SD card
  sudo ln -s /media/2p /2p               # for the 2nd SD card
  sudo ln -s /home/$USER /h              # home directory
  sudo ln -s /1p/pandora/apps /a         # 'apps' PNDs (on desktop and menu)
  sudo ln -s /1p/pandora/menu /p         # 'menu' PNDs (only in menu)
  sudo ln -s /1p/pandora/appdata /d      # appdata for each PND: settings &c
  sudo ln -s /mnt/utmp /u                # PNDs mounted here (with appdata)
  sudo ln -s /u/codeblocks /cb           # shortcut to code::blocks
  sudo ln -s /1p/code /c                 # shortcut to your code projects
  sudo ln -s /1p/code/x /x               # shortcut to your scripts

Install dev tools

• code::blocks comes with many compilers and libs, it’s large: almost 1GB in size
• we will be using the command-line.  Witness the power of Linux!
• Download code::blocks to your PC, then copy to pandora/apps on your SD card
• Code::blocks dev libs are in /mnt/utmp/codeblocks/usr/lib aka /cb/usr/lib
• Fetch a few helpful scripts that I wrote: cbdev, pnd, qos, optimize

  cd /x
  wget http://sam.nipl.net/pnd-x/cbdev  # use code::blocks in this tty
  wget http://sam.nipl.net/pnd-x/pnd    # build a pnd
  wget http://sam.nipl.net/pnd-x/qos    # wifi, be reliable
  chmod +x *
  wget http://sam.nipl.net/pnd-x/optimize   # gcc optimizer settings

Start code::blocks cli

• Run the code::blocks cli (command line interface) by clicking the icon
• Enter your password
• Now ssh in from your PC
• Run my cbdev script – this lets you use the code::blocks tools in this ssh terminal

Get source code for the app

• Code::blocks comes with many tools, including:  git, hg, svn for code management
• I used hg (mercurial) to fetch the latest dillo source code:

  cd /c    # /media/1p/code
  hg clone http://hg.dillo.org/dillo dillo3

• For a large git project, use git clone --depth 1 repo-url to skip the history
• If you think the latest code might be wonky, use a stable release archive.

Fetch and build needed libs

• Reading the instructions for building Dillo, we can see it uses the FLTK 1.3 library (fast-light toolkit).  Luckily, this is already included with code::blocks.
• For some apps, you will need to build libs they require.  We don’t need to do this for Dillo.  When building libraries, we want to install them into the code::blocks appdata:

  ./configure --prefix /mnt/utmp/codeblocks/usr    # and other options

Try to build the app

• I switch from 600Mhz to 1GHz when compiling. I don’t believe it does any harm
• It can take a few tries, to find the right settings.  Unleash your Linux power!

• Set the install prefix to /mnt/utmp/dillo3 – the pnd will run here
• We want their “experimental” https support, so --enable-ssl
• Optimize for Pandora, there is a thread on optimization here, see also the gcc doc.
  • -O3 means “strongly optimize”
  • -mcpu=cortex-a8, optimize for our CPU, the code won’t work on a different ARM
  • -mfpu=neon, use the neon SIMD floating point unit
  • -mfloat-abi=softfp, for hardware FPU code, with soft-float calling conventions
  • -ffast-math, do math fast! and don’t worry if it’s out by 1e-12 or whatever
• My first build failed with link errors, I had to link these extra libraries: -ldl -liconv
• So, here’s how I configure and build Dillo … this goes in a script:

  cd /c/dillo3
  CFLAGS="-O3 -mcpu=cortex-a8 -mfpu=neon -mfloat-abi=softfp -ffast-math"
  LIBS="-ldl -liconv"
  ./configure --prefix=/mnt/utmp/dillo3 CFLAGS="$CFLAGS" CXXFLAGS="$CFLAGS" LIBS="$LIBS" --enable-ssl

• When it configures okay, time to run make … and wait a while:

  make

Install the app

• We run make install, to collect what will go in the PND.  This is not always needed.
• We don’t want to install to the NAND, so let’s make a temporary symlink from /mnt/utmp/dillo3 to /media/1p/install/dillo3 on the SD card.  We can then install Dillo to the place it will run.

  cd /1p/install
  mkdir dillo3
  sudo ln -s $PWD/dillo3 /mnt/utmp/

• For Dillo, following their instructions, we will run make install-strip. This installs everything, then strips debugging symbols from the executables to make them smaller. For other apps, we can run make install ; strip executable to do that.

  cd /c/dillo3
  make install-strip

• Now Dillo is installed at /mnt/utmp/dillo3, or really: /media/1p/install/dillo3
• Let’s do a quick test and run the installed dillo – it should open a Dillo browser window

  cd /u/dillo3
  bin/dillo

• Hooray, it works!!

Make special changes for Pandora

• This is the real “porting” bit, I didn’t need to do much for Dillo!
• I edited dillorc in the source tree and changed settings to use small icons, etc
• Other ideas (for other apps / games): make them use the d-pad, ABXY buttons, shoulder buttons, nubs, sdl-omap / hardware scaler, etc.
• After writing all this, I made some more changes, so Dillo works better on Pandora
• The appdata union for PNDs doesn’t work well with FAT32, hacks may be needed. In short, we can’t add or change files in sub-dirs, need to make new dirs in the union fs.

More than you ever wanted to know about the PND system

• A PND is a read-only file system image, with XML info and a PNG icon appended to it
• Most PNDs use squashfs, for compression.  iso9660 (CD-ROM) filesystem is also used
• Each PND has an “appdata” folder, for config, game data, saves, bookmarks…
• The dillo3 appdata will be at /media/1p/pandora/appdata/dillo3
• When you run a PND, first the PND image itself is mounted …
• … Then an ‘aufs‘ union is mounted, putting the appdata on top the PND files
• The mounted PND filesystem is writable, has the PND files with appdata over the top
• Any changes to the mounted PND filesystem are written to the appdata
• At least, that’s how it works in theory…
• …But when appdata is on a FAT32 SD card, it does not quite work :/
• In case of this FAT32 bollocks, we cannot write files in the PND’s sub-folders
• If we need to write to a sub-folder, we must create a new one in the union / appdata
• This can be done carefully in the run.sh script.  we won’t do it for this Dillo example

Prepare PXML, icon, run script, screenshots

• I suggest a short, semi-compliant PXML.xml
• You can use the full <application> / <package> business, but it’s extra pain for no gain!

  cd /c/dillo3
  wget http://pandoria.org/src/beebem/PXML.xml  # get a nice short PXML file
  vi PXML.xml                                   # change it for your app
  cp PXML.xml /1p/install/dillo3/  # copy the PXML to our install dir

• Find or make an icon:

  wget http://www.dillo.org/Icons/d2_icon3_122x122.png -O icon.png
  cp icon.png /1p/install/dillo3/

• Write a run.sh script to launch the program:

  #!/bin/sh
  set -a
  PATH=$PWD/bin:$PATH
  LD_LIBRARY_PATH=$PWD/lib:$LD_LIBRARY_PATH
  exec dillo "$@"

  cp run.sh /1p/install/dillo3/

  • set -a exports all shell variables, so child processes see them
  • The script should run from /mnt/utmp/dillo3
  • We change PATH to include the dillo bin dir at /mnt/utmp/dillo3/bin
  • We change LD_LIBRARY_PATH to use our “shared” libs in /mnt/utmp/dillo3/lib
  • We exec dillo, passing along any command-line args (there won’t be any, but hey)
  • Using “exec” prevents the shell script from hanging around waiting
  • For a simpler app, you could just do like: exec ./hello

• You can make screenshots with SnapSnap.  Sexy, so your app will be popular!

• Be sure to mention run.sh, icon.png and the screenshots in your PXML.xml file!  it’s easy if you use the same simple names for each PND, don’t call them like: run-dillo.sh, dillo-icon.png…

Build the pnd

• Finally we get to make the pnd!  this is the easy bit:

  cd /1p/install/dillo3/
  /usr/sbin/mksquashfs . ../dillo3.pnd -noappend -all-root
  cat PXML.xml icon.png >>../dillo3.pnd

• You can use my one-line pnd script, it does that automatically
• Don’t use /usr/pandora/scripts/pnd_make, it’s long and less flexible too …
• You might build some PNDs from a source tree; use like -e .git to exclude a folder
• Install the PND, so it shows up on your desktop

  sudo rm /mnt/utmp/dillo3
  mv ../dillo3.pnd /a/

Test and fix the pnd

• Now we need to test if it works. Nothing ever works the first time!
• Try to run Dillo after a reboot (code::blocks not mounted).
• If it doesn’t work, check the log /tmp/pndrun_dillo3.out for an error
• Look between the mount and un-mount messages
• For Dillo, it needs some libs from code::blocks.  I ran ldd to check on this:

  cd /1p/install/dillo3
  ldd ./dillo

• ldd lists the shared libraries a program needs. It said some libraries were “not found”.
• I ran code::blocks again, and copied across those “not found” libraries.

  cd /cb/usr/lib/
  cp libiconv.so.2 libjpeg.so.8 libpng14.so.14 libstdc++.so.6 /1p/install/dillo3/lib/

• I rebuilt the PND, reboot, and test again … and now it’s working!  Yay 🙂
• After a few builds, it helps to automate the install -> pnd steps, I am using this script

Distribute your port

• Upload the PND to your server
• Alpha test on irc – /join #openpandora @ irc.freenode.net
• This is more fun than testing it yourself!

  rsync ../dillo3.pnd yourserver.net:www/

• I can provide web space if you need some
• Don’t forget to upload the source code, or include it in the PND
• Upload your PND to the repo – mention it’s a “beta” and ask for mercy!
• Post about your PND in Software News on the boards
• Mention it’s a “beta” in the thread, you can edit the title later in “full editor”
• You could make a thread before you release, to get help and advice

Wrapping up

Well, that’s the end of this long guide.  We set up our compilers & dev kit, built a browser direct on the Pandora, put it in a PND, and shared it with everyone.  Good stuff!

The next stage is to make fixes and changes, improving the build until it’s works really well.  People will post suggestions on the forum thread. Try to follow up quickly, and ask when you need help. My first port was BeebEm… one user waited patiently for many months while I procrastinated and did other stuff, instead of asking for the help I needed to fix it!

Here’s hoping this post can help you with coding and porting for the Pandora.

When you port some great app like Dillo, you will feel very happy – like these lucky girls who got to wear a special bra with wings.  Porting apps to Open Pandora makes you feel like that.

— Sam aka DREDD