The default Blackfin development board Linux and User-Space configuration examples, are optimized for ease of use and flexibility, and not for size and startup time.

There are hundreds of options enabled by default, allowing you to test drive various Linux drivers and application examples. Or there are options only useful during development and debugging. Having these options in an embedded production system is pointless.

Small memory footprint, low power and a fast boot time are typically in the interest of the embedded system designer.

This capture illustrates a 1 Second boot. For instrumentation I used GPIOs.

Configuration:

• Linux-2.6.24
• Standard kernel features including MTD/Block Layer support, but without TCP/IP Networking and kernel debug facilities.
• Rootfs in Initial RAM filesystem and RAM disk (initramfs/initrd).

Config files used during this benchmark are available here: 1 Sec. Boot Config Files

Configuration:

• Linux-2.6.24
• Standard kernel features including MTD/Block Layer support, TCP/IP Networking but without kernel debug facilities.
• Rootfs in Initial RAM filesystem and RAM disk (initramfs/initrd).

Config files used during this benchmark are available here: 1 Sec. Boot TCPIP Config Files

• Increase SCLK and CCLK in u-boot (don’t forget to adjust the SDRAM timing parameters). Typical values for BF537/527 set CCLK to 525MHz while SCLK is 131.5MHz.

• When boot speed is a major concern, and you are trying to save a few milliseconds- avoid using SPI Flash (BFIN_BOOT_SPI_MASTER) preferably use NOR Flash, there are now generic drivers for GPIO assisted maps in u-boot and the Linux kernel allowing you to utilize more memory than actually mapped by physical address lines. With latest u-boot (starting SVN revision 1326) there is DMA SPI Flash Read support – this boosts the sequential read speed up to 3.2MByte/s.

See SPI flash boot example below: spi_boot_linux_form_processor_reset_into_user_space_in_less_than_1.4_second

• When booting from NOR Flash:

Set CONFIG_BFIN_BOOT_MODE to BFIN_BOOT_PARA instead of BFIN_BOOT_BYPASS this saves approx. 2ms

• Important - tune your EBIU timing parameters according to your NOR flash timing specification, the default values are not tuned.

• If you are hunting for ms - Remove unused u-boot features and commands, this will decrease the size of u-boot and therefore speedup the time required for loading u-boot into RAM. Above example used the default BF537-STAMP u-boot configuration, except I increased SCLK to 131.5MHz, adjusted the EBIU Async Memeory timing and removed the RTC support.

See also here: env

u-boot env:
#set bootdelay 0

Compile u-boot with #define CONFIG_SILENT_CONSOLE

u-boot env:
#set silent 1

u-boot env:
#set baudrate 115200

The time spend here scales linearly with the size of your image your loading into memory.

Image sizes in the example discussed here:

michael@mhenneri-D02:~/devel/svn/uclinux-dist-trunk_3/images> ll
total 9600
lrwxrwxrwx 1 michael users      18 2008-08-05 13:12 linux -> linux.initramfs.gz
-rwxr-xr-x 1 michael users 2185567 2008-08-05 13:12 linux.initramfs
-rwxr-xr-x 1 michael users 1816927 2008-08-05 13:12 linux.initramfs.gz
-rw-r--r-- 1 michael users  773120 2008-08-05 13:12 rootfs.initramfs
-rw-r--r-- 1 michael users    8236 2008-08-05 13:12 rootfs.initramfs.contents
-rw-r--r-- 1 michael users  406936 2008-08-05 13:12 rootfs.initramfs.gz
-rw-r--r-- 1 michael users  359142 2008-08-05 13:12 System.map.initramfs
-rw-r--r-- 1 michael users  359142 2008-08-05 13:12 System.map.initramfs.gz
lrwxrwxrwx 1 michael users      16 2008-08-05 13:12 uImage -> uImage.initramfs
-rw-r--r-- 1 michael users  940474 2008-08-05 13:12 uImage.initramfs
-rw-r--r-- 1 michael users  941208 2008-08-05 13:12 uImage.initramfs.gz
-rw-r--r-- 1 michael users  533519 2008-08-05 13:12 vmImage
-rwxr-xr-x 1 michael users 1398334 2008-08-05 13:12 vmlinux

• Preferably use compressed images (uImage, vmImage), especially when loading from slow Flash memory.
• Disable the checksum calculation over the complete image in the bootm command to trade speed for safety in the boot process. Note that the header checksum is still verified.

u-boot env:
#set verify n

• Remove unused kernel features and functionality. Kernel debugging is to late in production systems, therefore remove all debug functionality.

See 1 Sec. Boot Config Files this configuration can be booted in exactly 1 Second.

• Pass loops_per_jiffy on the kernel command line lpj=xxxxxxx to the kernel. (get the value once from your kernel startup messages)

Example:
Calibrating delay loop... 1046.52 BogoMIPS (lpj=2093056)

• Disable earlyprintk and suppress kernel messages on startup see: silent_booting

Example Kernel command line:

      root=/dev/mtdblock0 rw console=ttyBF0,115200 quiet lpj=2093056

Further speedups are easily possible:

• If your application doesn’t need to access mtd/storage other then your initial ram filesystem (initramfs/initrd), remove the block and mtd layer (approx -100ms)

By removing more options, such as support for printk, System V IPC, etc. the overall bootime can be less than 0.7 Seconds.

• Slim down your user space configuration by removing unused commands in busybox and by disabling other applications in the uClinux-dist.

• Try to find the executable file format that fits best your application needs: FLAT, Shared-FLAT or FDPIC. Uncheck installation of shared libraries if you are not going to use them. Typically Shared-FLAT performs best.

• Don’t build debugable libraries and applications: remove the –g flag form your user/lib CFLAGS.

• Remove unused functionality from your /etc/rc script. For example don’t try to mount usbfs if you are not planning to use USB, etc.

• Use static device nodes (/dev/) versus dynamic. Static device nodes don’t have any runtime overhead.

• Remove options form your C-library you are not going to use such as wchar support, etc.

This capture illustrates a 0.4 Seconds boot, of minimalistic system.

Almost all device drivers are removed except for Serial and GPIO. Kernel features such as loadable module support and many others are removed. User space only includes init, shell, ls and echo.

This setup may fit together with the bootloader into 0.5 Mega Byte of flash memory.

The config files used can be downloaded here:

Config Files

michael@mhenneri-D02:~/devel/svn/uclinux-dist-trunk_3/linux-2.6.x> bfin-uclinux-size vmlinux */built-in.o
   text    data     bss     dec     hex filename
 713068   54280   17604  784952   bfa38 vmlinux
  69558    3692    3428   76678   12b86 drivers/built-in.o
 116032    1224    1748  119004   1d0dc fs/built-in.o
  13820    2509    1128   17457    4431 init/built-in.o
  95120    7176    5488  107784   1a508 kernel/built-in.o
  19028      12      16   19056    4a70 lib/built-in.o
  44700    2852     508   48060    bbbc mm/built-in.o
    120       0       0     120      78 net/built-in.o
   1352       4       4    1360     550 security/built-in.o
 305664       0       0  305664   4aa00 usr/built-in.o

michael@mhenneri-D02:~/devel/svn/uclinux-dist-trunk_3/images> ll
total 4180
lrwxrwxrwx 1 michael users     18 2008-08-05 15:16 linux -> linux.initramfs.gz
-rwxr-xr-x 1 michael users 939908 2008-08-05 15:16 linux.initramfs
-rwxr-xr-x 1 michael users 796548 2008-08-05 15:16 linux.initramfs.gz
-rw-r--r-- 1 michael users 305664 2008-08-05 15:16 rootfs.initramfs
-rw-r--r-- 1 michael users   6895 2008-08-05 15:16 rootfs.initramfs.contents
-rw-r--r-- 1 michael users 164813 2008-08-05 15:16 rootfs.initramfs.gz
-rw-r--r-- 1 michael users 125155 2008-08-05 15:16 System.map.initramfs
-rw-r--r-- 1 michael users 125155 2008-08-05 15:16 System.map.initramfs.gz
lrwxrwxrwx 1 michael users     16 2008-08-05 15:16 uImage -> uImage.initramfs
-rw-r--r-- 1 michael users 415208 2008-08-05 15:16 uImage.initramfs
-rw-r--r-- 1 michael users 416145 2008-08-05 15:16 uImage.initramfs.gz
-rw-r--r-- 1 michael users 250272 2008-08-05 15:16 vmImage
-rwxr-xr-x 1 michael users 623513 2008-08-05 15:16 vmlinux

This example uses a similar configuration than the first 1 Sec. boot from NOR flash example.

Layout: 3 MTD partitions on SPI Dataflash:

Configuration:

• Standard kernel features including MTD/Block Layer support, but without TCP/IP Networking and kernel debug facilities.
• JFFS2 Rootfs in SPI Flash.
• Static Device Nodes

The config files used can be downloaded here:

Config Files No NET

michael@mhenneri-D02:~/devel/svn/uclinux-dist-trunk_3/linux-2.6.x> bfin-uclinux-size vmlinux */built-in.o
   text    data     bss     dec     hex filename
 949649   64356   49016 1063021  10386d vmlinux
  32606     962    1080   34648    8758 block/built-in.o
 121916    4794    3828  130538   1fdea drivers/built-in.o
 302511    2560    2132  307203   4b003 fs/built-in.o
  16807    2509    1128   20444    4fdc init/built-in.o
  14176     684       0   14860    3a0c ipc/built-in.o
 155093   12968   30476  198537   30789 kernel/built-in.o
  35164      72    2296   37532    929c lib/built-in.o
  71212    3740     576   75528   12708 mm/built-in.o
 119772    3972    1796  125540   1ea64 net/built-in.o
   2522       0       0    2522     9da security/built-in.o
    133       0       0     133      85 usr/built-in.o

lrwxrwxrwx 1 michael users      18 2008-08-21 17:26 linux -> linux.initramfs.gz
-rwxr-xr-x 1 michael users 2130338 2008-08-21 17:26 linux.initramfs
-rwxr-xr-x 1 michael users 1827234 2008-08-21 17:26 linux.initramfs.gz
-rw-r--r-- 1 michael users  643072 2008-08-21 17:25 rootfs.initramfs
-rw-r--r-- 1 michael users   12071 2008-08-21 17:25 rootfs.initramfs.contents
-rw-r--r-- 1 michael users  343870 2008-08-21 17:25 rootfs.initramfs.gz
-rw-r--r-- 1 michael users  430228 2008-08-21 17:25 rootfs.jffs2
-rw-r--r-- 1 michael users  377012 2008-08-21 17:26 System.map.initramfs
-rw-r--r-- 1 michael users  377012 2008-08-21 17:26 System.map.initramfs.gz
lrwxrwxrwx 1 michael users      16 2008-08-21 17:26 uImage -> uImage.initramfs
-rw-r--r-- 1 michael users  915303 2008-08-21 17:26 uImage.initramfs
-rw-r--r-- 1 michael users  916128 2008-08-21 17:26 uImage.initramfs.gz
-rw-r--r-- 1 michael users  571390 2008-08-21 17:25 vmImage
-rwxr-xr-x 1 michael users 1473255 2008-08-21 17:25 vmlinux

Configuration:

• Standard kernel features including MTD/Block Layer support, with TCP/IP Networking but without kernel debug facilities.
• JFFS2 Rootfs in SPI Flash.
• Static Device Nodes

The config files used can be downloaded here:

Config Files NET

michael@mhenneri-D02:~/devel/svn/uclinux-dist-trunk_3/linux-2.6.x> bfin-uclinux-size vmlinux */built-in.o
   text    data     bss     dec     hex filename
1982319   74400   55416 2112135  203a87 vmlinux
  32606     962    1080   34648    8758 block/built-in.o
   3299      36       8    3343     d0f crypto/built-in.o
 136690    5270    3860  145820   2399c drivers/built-in.o
 302511    2560    2132  307203   4b003 fs/built-in.o
  16807    2509    1128   20444    4fdc init/built-in.o
  14176     684       0   14860    3a0c ipc/built-in.o
 155093   12968   30476  198537   30789 kernel/built-in.o
  35164      72    2296   37532    929c lib/built-in.o
  71212    3740     576   75528   12708 mm/built-in.o
 332314   13772    8164  354250   567ca net/built-in.o
   2522       0       0    2522     9da security/built-in.o
 803840       0       0  803840   c4400 usr/built-in.o

lrwxrwxrwx 1 michael users      18 2008-08-21 16:24 linux -> linux.initramfs.gz
-rwxr-xr-x 1 michael users 2597785 2008-08-21 16:25 linux.initramfs
-rwxr-xr-x 1 michael users 2220953 2008-08-21 16:24 linux.initramfs.gz
-rw-r--r-- 1 michael users  803840 2008-08-21 16:24 rootfs.initramfs
-rw-r--r-- 1 michael users   12783 2008-08-21 16:24 rootfs.initramfs.contents
-rw-r--r-- 1 michael users  429021 2008-08-21 16:24 rootfs.initramfs.gz
-rw-r--r-- 1 michael users  529440 2008-08-21 16:24 rootfs.jffs2
-rw-r--r-- 1 michael users  432847 2008-08-21 16:25 System.map.initramfs
-rw-r--r-- 1 michael users  432847 2008-08-21 16:24 System.map.initramfs.gz
lrwxrwxrwx 1 michael users      16 2008-08-21 16:25 uImage -> uImage.initramfs
-rw-r--r-- 1 michael users 1142481 2008-08-21 16:25 uImage.initramfs
-rw-r--r-- 1 michael users 1141928 2008-08-21 16:24 uImage.initramfs.gz
-rw-r--r-- 1 michael users  713205 2008-08-21 16:24 vmImage
-rwxr-xr-x 1 michael users 1775055 2008-08-21 16:24 vmlinux

For more information see here: Serial NOR Flash and U-Boot

• Set SCLK = 131.5 MHz
• Set SPI_CLK = 32.875 MHz
• Enable Support for Silent Console
• Disable RTC support in u-boot

Index: include/configs/bf537-stamp.h
===================================================================
#define CONFIG_BFIN_BOOT_MODE			BFIN_BOOT_SPI_MASTER
#define CONFIG_VCO_MULT				21
#define CONFIG_SCLK_DIV				4

#define CONFIG_EBIU_SDRRC_VAL  			0x3F8
#define CONFIG_EBIU_SDGCTL_VAL 			0x009111CD

#define CONFIG_SPI
#define CONFIG_SPI_BAUD				2
#define CONFIG_SPI_BAUD_INITBLOCK		2

#undef CONFIG_RTC_BFIN
#define CONFIG_SILENT_CONSOLE

baudrate=57600
loads_echo=1
ipaddr=192.168.0.15
serverip=192.168.0.2
autoload=no
rootpath=/romfs
gatewayip=192.168.0.1
netmask=255.255.255.0
hostname=bf537-stamp
loadaddr=0x1000000
ubootfile=u-boot.ldr
update=tftp $(loadaddr) $(ubootfile);eeprom write $(loadaddr) 0x0 $(filesize)
addip=set bootargs $(bootargs) ip=$(ipaddr):$(serverip):$(gatewayip):$(netmask):$(hostname):eth0:off
ramargs=set bootargs root=/dev/mtdblock0 rw earlyprintk=serial,uart0,57600 console=ttyBF0,57600
ramboot=tftp $(loadaddr) uImage;run ramargs;run addip;bootm
nfsargs=set bootargs root=/dev/nfs rw nfsroot=$(serverip):$(rootpath),tcp,nfsvers=3
nfsboot=tftp $(loadaddr) vmImage;run nfsargs;run addip;bootm
flashboot=bootm 0x20100000
ethaddr=00:E0:22:FE:52:3A
ethact=Blackfin EMAC
verify=n
bootcmd=eeprom read 0x1000000 0x40000 8b7ff;bootm
silent=y
stdin=serial
stdout=nc
stderr=nc
bootargs=root=/dev/mtdblock2 rw rootfstype=jffs2 console=ttyBF0,57600 quiet lpj=2093056
bootdelay=0

root:/> cat /etc/rc
# This file targets a *development* system.  Many things are not
# needed for *production* systems.  Comment out what you do not need.

#hostname blackfin

mount -t proc proc /proc -o noexec,nosuid,nodev
mount -t sysfs sysfs /sys -o noexec,nosuid,nodev

#
# Setup dynamic /dev and let mdev manage it for us
#
#if [ -e /bin/mdev ] ; then
#       mount -t tmpfs mdev /dev -o exec,nosuid,mode=0755,size=10M
#       [ -e /proc/sys/kernel/hotplug ] && echo /bin/mdev > /proc/sys/kernel/hotplug
#       /bin/mdev -s
#fi

#
# Setup dynamic pseudo-terminal file system ... needed by any system that
# uses telnet or ssh (like dropbear) rather than the old static pty system.
# You should give this a dedicated gid and set the mode to 0650.
#
mkdir -p /dev/pts
mount -t devpts devpts /dev/pts -o noexec,nosuid

#
# Give the log/state/runtime files in /var temporary storage that'll be
# lost when we reboot.
#
mount -t ramfs var /var
mkdir /var/tmp /var/log /var/run /var/lock

#
# Give temp files a read/write location not in flash/etc...
#
mount -t tmpfs tmp /tmp -o nosuid,nodev

#
# Needed if you plan on doing anything USB related
#
#[ -d /proc/bus/usb ] && mount -t usbfs usbfs /proc/bus/usb

#
# Needed if you run an NFS server on the board
#
#grep -qs nfsd /proc/filesystems && mount -t nfsd nfsd /proc/fs/nfsd

#
# Useful for debugging only
#
#[ -d /sys/kernel/debug ] && mount -t debugfs debugfs /sys/kernel/debug

#
# Create your own interpreters for random file formats
#
#[ -d /proc/sys/fs/binfmt_misc ] && mount -t binfmt_misc binfmt_misc /proc/sys/fs/binfmt_misc

#
# User interface to security modules (selinux, etc...)
#
#[ -d /sys/kernel/security ] && mount -t securityfs securityfs /sys/kernel/security

#
# Do random network setup
#
#dhcpcd &
#ifconfig eth0 10.100.4.251 up
#ifconfig lo 127.0.0.1
#inetd &

#cat /etc/issue
#cat /etc/motd

Linux version 2.6.26.2-ADI-2009R1-pre-svn5184-dirty8 (michael@mhenneri-D02) (gcc version 4.1.2 (ADI svn)) #5648 Thu Aug 21 16:19:22 CEST 2008
Board Memory: 64MB
Kernel Managed Memory: 64MB
Memory map:
  fixedcode = 0x00000400-0x00000490
  text      = 0x00001000-0x000e6dd0
  rodata    = 0x000e6de0-0x00111da8
  bss       = 0x00111dc0-0x0011f638
  data      = 0x0011f638-0x0012e000
    stack   = 0x0012c000-0x0012e000
  init      = 0x0012e000-0x00141000
  available = 0x00141000-0x03eff000
  DMA Zone  = 0x03f00000-0x04000000
Hardware Trace Active and Enabled
Blackfin support (C) 2004-2008 Analog Devices, Inc.
Compiled for ADSP-BF537 Rev 0.3
Blackfin Linux support by http://blackfin.uclinux.org/
Processor Speed: 525 MHz core clock and 131 MHz System Clock
 boot memmap: 0000000000141000 - 0000000003eff000 (usable)
On node 0 totalpages: 16127
  DMA zone: 126 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 16001 pages, LIFO batch:3
  Normal zone: 0 pages used for memmap
  Movable zone: 0 pages used for memmap
NOMPU: setting up cplb tables for global access
Instruction Cache Enabled
Data Cache Enabled (write-back)
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 16001
Kernel command line: root=/dev/mtdblock2 rw rootfstype=jffs2 console=ttyBF0,57600 quiet lpj=2093056
Configuring Blackfin Priority Driven Interrupts
PID hash table entries: 256 (order: 8, 1024 bytes)
console [ttyBF0] enabled
Dentry cache hash table entries: 8192 (order: 3, 32768 bytes)
Inode-cache hash table entries: 4096 (order: 2, 16384 bytes)
Kernel managed physical pages: 16127
Memory available: 62644k/65536k RAM, (76k init code, 919k kernel code, 285k data, 1024k dma, 584k reserved)
Calibrating delay loop (skipped)… 1046.52 BogoMIPS preset
Mount-cache hash table entries: 512
Blackfin Scratchpad data SRAM: 4 KB
Blackfin L1 Data A SRAM: 16 KB (15 KB free)
Blackfin L1 Data B SRAM: 16 KB (16 KB free)
Blackfin L1 Instruction SRAM: 48 KB (42 KB free)
net_namespace: 192 bytes
NET: Registered protocol family 16
Blackfin GPIO Controller
Blackfin DMA Controller
stamp_init(): registering device resources
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 2048 (order: 2, 16384 bytes)
TCP bind hash table entries: 2048 (order: 1, 8192 bytes)
TCP: Hash tables configured (established 2048 bind 2048)
TCP reno registered
NET: Registered protocol family 1
JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
msgmni has been set to 122
io scheduler noop registered (default)
simple-gpio: now handling 48 GPIOs: 0 - 47
Serial: Blackfin serial driver
bfin-uart.1: ttyBF0 at MMIO 0xffc00400 (irq = 18) is a BFIN-UART
bfin_mac_mdio: probed
bfin_mac: attached PHY driver [Generic PHY] (mii_bus:phy_addr=0:01, irq=-1, mdc_clk=2500000Hz(mdc_div=25)@sclk=131MHz)
bfin_mac bfin_mac.0: Blackfin on-chip Ethernet MAC driver, Version 1.1
mtd_dataflash spi0.1: at45db321d (4096 KBytes) pagesize 512 bytes, erasesize 4096 bytes
Creating 3 MTD partitions on “SPI Dataflash”:
0x00000000-0x00040000 : “bootloader(spi)”
0x00040000-0x00120000 : “linux kernel(spi)”
0x00120000-0x00400000 : “file system(spi)“
bfin-spi bfin-spi.0: Blackfin BF5xx on-chip SPI Controller Driver, Version 1.0, regs_base@ffc00500, dma channel@7
rtc-bfin rtc-bfin: rtc core: registered rtc-bfin as rtc0
bfin-wdt: initialized: timeout=20 sec (nowayout=0)
TCP cubic registered
NET: Registered protocol family 17
rtc-bfin rtc-bfin: setting system clock to 1970-01-01 21:06:10 UTC (75970)
JFFS2 write-buffering enabled buffer (512) erasesize (8192)
VFS: Mounted root (jffs2 filesystem).
Freeing unused kernel memory: 76k freed
dma_alloc_init: dma_page @ 0x0013a000 - 256 pages at 0x03f00000

Loading compressed kernel (vmImage) or kernel+initrd (uImage) in u-boot from SPI flash is fast. However accessing SPI flash backed JFFS2 file systems in the kernel tends to be slow.

Therefore it is recommendable loading a kernel image with an attached Initial RAM Disk(initrd).

Then mount a JFFS2 file system (from your rc script), holding files that needs to be preserved during reboots, of files that are not frequently accessed. (Sacrifice runtime memory usage for boot speed.)
Use RamFS or TmpFS File systems for temporary files not needed after a reboot.