The Linphone porting project for BF533/7 STAMP

Voice communication, as we use it in the modern age, was developed by Alexander Graham Bell in 1876. Since then, the Telephone has not changed much. We still pick up the handset, dial a number (sometimes by voice), and are connected to a person at the other end. Almost nothing but style and color has changed in the user interface of a phone since then.

Using the internet as a phone service, may seem like Star Trek to some, but Voice over Internet Protocol (VoIP) began as the result of work done by some hobbyists in early 1995 when PC-to-PC communication was available. Later that same year, commercial software was released to run on a home PC (486/33 MHz) with sound cards, speakers, microphone, and modem. The software compressed the voice signal, translated it into voice packets, and shipped it out over the Internet. The technology worked as long as both the caller and the receiver had the same equipment and software. Although the sound quality was nowhere near that of conventional equipment at the time, this effort represented the first VoIP phone.

In the past 10 years, many things have happened:

• processors are more powerful - and cheaper
• The Internet has expanded - IDC and IWS say that in December of 1995 the Internet had 16 million users, and in June of 2006 is has 1,023 million (15.7% of the world population) .
• Broadband has become more affordable, and is rolled out to many more people. Jupiter Research predicted that up to 69 million households - 78 percent of U.S. online homes - would have broadband by 2010 if providers continue to lower their prices.
• VoIP standards have developed around Real-time Transfer Protocol (RTP) and Session Initiation Protocol (SIP)
• Voice Compression Codecs¹⁾, standardize on International Telecommunications Union (ITU) codecs, including but not limited to G.711, G.722, G.723.1, G.726, and G.729.
• Open Source, and IP free Voice codecs become available, which are very good. See a comparison.
• Open Source applications like Linphone, Gnomemeeting, and Asterisk allow people to break away from the PSTN.
• Skype²⁾ shows people end users want VoIP if it all works - over 150 Million Downloads, and 12.5 billion minutes (~23,766 years) have been served.

Taking advantage of all these things, the developers who were working on Blackfin/uClinux ported the necessary components to the Blackfin/uClinux to allow you to make a phone call.

• Speex
• oRTP (which comes as part of Linphone, because it's from the same author)
• oSIP
• Linphone

All four modules have been incorporated into the Blackfin/uClinux SVN, and should build properly by following the directions below.

Linphone lets you phone to your friends anywhere in the whole world, freely, simply by using the internet. The cost of the phone call is the cost that you spend connected to the internet. It is a SIP (Session Initiation Protocol,rfc3261) compatible VoIP telephone application. As a SIP phone application it can call and receive calls from most of the SIP compatible hardware and software implementations. Such calls can either be direct, via a proxy or in most cases via a registrar, which is also able to route call from and into the PSTN (Public Switched Telephone Network). Such services are in a great variety available over the Internet. For the audio communication itself, Linphone supports the most commonly used CODECs G.711a/u, GSM, Speex, lpc10 and iLBC (the latter two currently not on Blackfin).

On a Linux host, it has a nice GUI (Gnome Desktop), and is available on the Blackfin as a console application linphonec.

Adding linphonec to the Blackfin/uClinux build system is very trivial.

1. Add audio support to the kernel (like the ad1836)
2. Add linphonec by selecting:

       Blackfin canned demos  ---> 
         [*] Linphone, a SIP compatible VoIP phone 

To setup, and run linphone follow these simple steps:

1. Configure networking
2. Execute linphonec from your shell. Then you will see a command prompt

   root:~> linphonec
   NLS disabled.
   Message:Found /dev/dsp.
   channel in masks set to 1023 = { 22 33 77 ff }
   Warning:Could not open /proc/modules.
   Message:Adding new codec PCMU/8000
   Message:Adding new codec GSM/8000
   Message:Adding new codec PCMA/8000
   Message:Adding new codec speex/8000
   Message:Adding new codec speex/16000
   Message:Adding new codec 1015/8000
   Ready.
   linphonec>

You should know, that the channel in masks message is from the kernel, which tells you, that linphonec switched the input from line-in to mic. The Adding new codec messages are only printed at the first start of linphonec. After that a .linphonec configuration file is created, which can be changed with a text editor.

You can :

• get help:

  linphonec> help
  Commands are:
  ---------------------------
        help      Print commands help
        call      Call a SIP uri
   terminate      Terminate the current call
      answer      Answer a call
       proxy      Manage proxies
   soundcard      Manage soundcards
        ipv6      Use IPV6
       refer      Refer the current call to the specified destination.
         nat      Set nat address
        stun      Set stun server address
    firewall      Set
   call-logs      Calls history
      friend      Manage friends
        play      play from a wav file
      record      record to a wav file
        quit      Exit linphonec
  ---------------------------
  Type 'help <command>' for more details.
  linphonec>

• call a user foo at a host 'bar' by typing

  linphonec> call sip:foo@bar

• To switch between OSS and ALSA sound drivers:

  linphonec> help soundcard
  'soundcard list' : list all sound devices.
  'soundcard use <index>' : select a sound device.
  'soundcard use files' : use .wav files instead of soundcard
  linphonec> soundcard list
  0: ALSA: default device
  1: ALSA: Analog Devices AD73311L
  2: OSS: /dev/dsp
  3: OSS: /dev/dsp
  linphonec> soundcard use 2
  Using sound device OSS: /dev/dsp
  linphonec> soundcard use 0
  Using sound device ALSA: default device

Enjoy!

Currently we enabled one way video in Linphone. To enable, in the configuration:

[*] Linphone, a SIP compatible VoIP phone 
[*]    Enable video 

Also please note linphone need to be built as FDPIC format.

• Setup ST vs6524 or vs6624 sensor as here.

  root:~> modprobe blackfin_cam

• Setup AD1836 sound card (AD1836 supports sample rate of 48KHz)

  root:~> modprobe snd-ad1836

• Start linphone, set it to capture video :

   
  root:~> linphone -C

• Display video: If you don't have another STAMP board with LCD module, you will need to install linphone (version later than linphone-1.7.1) on you Linux Host.

Linphone uses SDL for video display.

• Setup LCD LQ-035. Here we use landscape mode

root:~> modprobe bf537-lq035 landscape=1
 

Note that since the UD pin on LQ-035 board conflicts with SPI-CLK pin on AD1836A board. We have to cut off the UD pin of LQ-035 if to use AD1836 sound card. Also need to change the LQ-035 driver to undefine “UD” as this:

linux-2.6.x/drivers/video/bf537-lq035.c
//#define UD      GPIO_PF13     /* Up / Down */

• Linphone diplays video via SDL library. Please select “Virtual Terminal” option to make SDL work properly:

Character devices  ---> 
    [*] Virtual terminal 

• Setup sound card AD1836.

root:~> modprobe snd-ad1836

• Start Linphone for video display:

root:~> export SDL_NOMOUSE=1 // No mouse is required for SDL.
root:~> linphone -D

Video encoding is CPU consuming. Here we use PCMU or PCMA codec which cosumes less MIPS than speex. Also use AD73311 sound card which support 8MHZ sample rate (while AD1836 supports 48KHZ and requires software sample rate conversion).

In bellow .linphonerc configuration file, we select to use PCMU audio codec, and MPEG4 video codec. Also the bandwidth set to infinite (CIF at 17fps).

 
[net]
download_bw=0
upload_bw=0
firewall_policy=0

[video]
enabled=1
show_local=0

[audio_codec_0]
mime=speex
rate=16000
enabled=0

[audio_codec_1]
mime=speex
rate=8000
enabled=0

[audio_codec_2]
mime=PCMU
rate=8000
enabled=1

[audio_codec_3]
mime=PCMA
rate=8000
enabled=1

[video_codec_0]
mime=H263-1998
rate=90000
enabled=0

[video_codec_1]
mime=MP4V-ES
rate=90000
enabled=1

In most cases there are some adjustments to the linphonec configuration necessary (like adding a proxy, setting different ports, configuring preferred codecs,…). There are a few ways to do this.

First, the config file is named .linphonec and resides in ~/ which means the users homedirectory. If you use the root account for example, you'll find the .linphonec file in / which is the root users home directory. Of course it will only exists after it has been created from linphonec after its first invocation (if you can't find it, use ls -a ). For some configuration work, you can use linphonec directly (like adding a proxy) and for some other tasks you have to edit the config file (like setting a different port).

If you look at the config file, you'll see that it has basically the same format like the windows .ini files. This means it has different sections that are separated with an identifier in [] and between them there are some sort of variable declaration some_var=some_value. This is a very easy format, which could easily be edited.

You can now start to edit the file with an editor of your choice. This could be the vi under uClinux (if you have problems with the vi editor, there are numerous resources on the web. Just google for vi introduction) or you can copy the file to your PC (with ftp or rcp for example) and edit it there.

Another possibility would be to use the graphical linphone program under linux and copy the config file. If you have linux installed on a PC and have the linphone package installed you can execute linphone and set it up properly and test it. After everything works you can find the configuration file (for versions ⇐ 1.2) under ~/.gnome2/linphone. You can copy that file to your STAMP board (as .linphonec of course) or use it as template.

After you should test the new configuration and if it works, you can copy the .linphonec file into the romfs directory of your uClinux-dist. It will then be in all your new images you create. Due to a bu^H^Hfeature it won't even be deleted after a make clean.

Each time linphone is loaded, it will search for ”.linphonerc” config file. Each time linphone quits, it will store current configration into ”.linphonerc”. This is how the default .linphonerc config file looks (for linphone-1.6.0):

[net]
download_bw=0
upload_bw=0
firewall_policy=0


[sip]
sip_port=5060
guess_hostname=1
contact=sip:toto@unknown-host
use_info=0
use_ipv6=0
default_proxy=-1

[rtp]
audio_rtp_port=7078
video_rtp_port=9078
audio_jitt_comp=60
video_jitt_comp=60

[sound]
playback_dev_id=0
ringer_dev_id=0
capture_dev_id=0
rec_lev=80
play_lev=80
ring_lev=80
source=m
local_ring=/usr/share/sounds/linphone/rings/oldphone.wav
remote_ring=/usr/share/sounds/linphone/ringback.wav
echocancelation=0

[video]
enabled=0
show_local=0

[audio_codec_0]
mime=PCMU
rate=8000
enabled=1

[audio_codec_1]
mime=GSM
rate=8000
enabled=1

[audio_codec_2]
mime=PCMA
rate=8000
enabled=1

[audio_codec_3]
mime=speex
rate=8000
enabled=1

[audio_codec_4]
mime=speex
rate=16000
enabled=1

[audio_codec_5]
mime=1015
rate=8000
enabled=1

[video_codec_0]
mime=H263-1998
rate=90000
enabled=1

[video_codec_1]
mime=MP4V-ES
rate=90000
enabled=1

You can see six different sections ([net],[sip],[rtp],[sound],[video],[audio_codec_x]). There are also some additional sections, which are not in the default .linphonec config file ([proxy_x] and [auth_info_x]). They will be added if you define one or more proxies.

In the [net] section the general network parameters are defined.

• “download_bw” “upload_bw”: 0 - infinite (no limit on bandwidth). The unit is KB. These two values affects the bit rate of video/audio stream. The caller and callee side settings both affect video/audio bitrate and the rule is hard-coded in linphone. Here are some example settting:

 
The video display side setting affect capture side frame rate and frame size:
download_bw = 0, upload_bw = 0: CIF / 17 FPS
download_bw = 512, upload_bw = 512: CIF / 10 FPS
download_bw = 256, upload_bw = 256: QCIF / 10 FPS
download_bw = 128, upload_bw = 128: QCIF / 6 FPS

• The sip_port= parameter sets obviously the SIP port, which defaults to 5060. (RFC2543)
• The guess_hostname= parameter tells linphonec if it should guess the hostname or if it should just send the unmodified contact address in contact=. This parameter is only for direct connections and not for connections over an registrar. Also it might not necessarily work on uClinux.
• contact= defines the contact address of your linphonec SIP-Phone. Set it to sip:username@hostname.
• The use_info= parameter tells linphonec to use SIP info messages instead of RFC2833 (RFC2833) for DTMF tones.
• use_ipv6= is pretty self-explanatory.
• default_prox=x selects the proxy defined in proxy_x as default.

In this section you can select different RTP ports (for example if you want to use port redirection on your NAT-router, to make different linphone soft phones accessible from the Internet)

• audio_rtp_port=7078 defines the UDP port linphonec should use for the audio connection.
• video_rtp_port=9078 defines the UDP port linphonec should use for the video connection.
• audio_jitt_comp=60 defines the default audio jitter buffer size in ms.
• video_jitt_comp=60 the same for the video jitter buffer.

You should not change the RTP ports, unless you don't know what you are doing. If you have a private network with multiple computers behind a NAT router you can configure every linphone with different RTP ports and forward them from the NAT router to your ”linphone machines”. If you combine that with a dynamic DNS service and set the contact= field accordingly your linphonec could be accessible without a registrar or proxy even in a private network behind a NAT device with a dialup connection. But you should really know what you are doing, because there are several security and network stability implication you have to consider. (Not to mention, that you need to reconfigure the router and the router has to support it).

In the [sound] section you can configure some of the linphonec audio parameters, like the recording or output volume, the soundcard and the ringtones.

• With playback_dev_id=, ringer_dev_id= and capture_dev_id= you can select different soundcards and channels for playback, ringing and capture.
• rec_lev= sets the recording level.
• play_lev= sets the playback level.
• ring_lev= sets the ringtone level.
• source= selects the input source, where m is the microphone jack and l the line-in jack.
• local_ring= sets the audio file for the local ring. (The tone you hear, when your phone rings)
• remote_ring= sets the audio file for the remote ring (The callback tone)
• echocancelation= set as '1' to enable speex with echo cancellation.

There are two parameters enabled= and show_local= which are used to enable video support and to show the local video input (preview).

There are multiple [audio_codec_x] section, which define the available audio codecs and their properties. You can give a preference to a CODEC if you put it higher in the the CODEC list, but the CODEC which will be selected depends on the connection type (you can't use a G.771a/u CODEC which uses 64kBit/s not on a con_type=1 for example. Also the opposite side has to support the CODEC.

The format of the CODEC section looks like:

[audio_codec_0]
mime=PCMU
rate=8000
enabled=1

• Where [audio_codec_0] means, it is the first defined and therefor most preferred CODEC. The name of the mime type is PCMU which refers to G.711u. (PCMA to G.711a, speex to speex, GSM to GSM and 1015 to lpc10 which is theoretically available but not supported).
• The rate=8000 parameter sets narrow band. For wide band (with speex for example) you have to select rate=16000.
• enabled=1 tells linphonec that the CODEC is enabled. You can disable the CODEC with enabled=0. You should do this with the lpc10 CODEC.

For the description of the [proxy_x] and [auth_info_x] please refer to the next section Adding a proxy to linphonec.

You can add a proxy to your linphonec setup either with the proxy add command or with the linphone program on your host PC (and copy the [proxy_x] and [auth_info_x] sections later into your .linphonec file or you can edit the .linphonec file directly.

Currently the console client is working with the G.711a/u, GSM and Speex CODEC. It is able to make an receive calls either through a proxy/registrar or directly. It has been validated to work properly with the AD1836A cards and the AD73311 cards.