world leader in high performance signal processing
Trace: » processes

Development Processes

Every project, whether it is open or closed; hardware or software; electronics or digging a hole in the ground; all follow various processes. These processes ensure that things get tracked, completed, and finished on time. Processes typically save time. A process is just a means to an end - there are lots of good ones, and bad ones. We try to use the simplest/best process we can, while allowing everyone (developers and users) to get the jobs done - whether that be handing in that assignment for school, or release a project to production. (Although we don't like to do homework for people ;-) - to this end - we spend very little time on people's questions if they subscribed from a university, gmail, or hotmail - if you are from a real company who is making a commercial product, please register from that address, so we can help you better).

Every different open source project has their own processes, which most times are not well documented. This makes it difficult for occasional users to understand how to ask a question, or where to send a bug report. This section will try to address those questions, and document the processes that are followed on our project site.

Sometimes people want to go around the standard process, via sending a quick email to a developer. There is no such thing as a quick email. Normally - this only slows things down in the long run, since the developers :

  • stop developing to answer a question,
  • don't always follow up on things, since they are busy, and
  • does not tell everyone involved in the project there is a problem.

If you have a problem/question/bug - please follow the outlined process. If you are feeling ignored, or your bug is not being worked on, please understand that we work on things based on priority set by the impact of the issue. This either means a common bug that effects many people, or an issue that effects someone who is shipping millions of units. We are not unreasonable - if something is stopping you from shipping a product - let us know - we will try to help.

Overall, we are very much in the Bazaar catagory.

Documentation

All of our documentation for running on the Blackfin processor (whether it be the Linux kernel or user space applications, the Das U-Boot bootloader, or our GNU host tools) is done on this http://docs.blackfin.uclinux.org web site. Although there are many other sources of information for Linux, Das U-Boot, gcc and gdb, and Blackfin information on the internet - if you don't read it on this web site, may work - or it may not. This is especially true of Analog Devices Application Notes, or Engineer to Engineer Notes (EE-Note) about the Blackfin processor. Even though that information may be technically accurate - it may not reflect the way that Linux drivers expect hardware to be connected. This documentation site is the only authoritative place for Blackfin/Linux information, including hardware designs.

Although this is a wiki, write access is restricted to a small group of people. Every time someone makes a change to the documentation, a diff is emailed out to an open mailing list, where it is reviewed for clarity, technical accuracy, grammar and spelling. This ensures that the documentation is of the highest technical accuracy. Documentation is added based on a project member answering the same question multiple times on the forums, or when a developer adds new functionality.

If you would like to help volunteer to edit this wiki, add pages, fix mistakes, or draw better pictures, you can check that your additions will meet the requirements of the copyright information. If you just want to point out bugs in the documentation, we like that too.

Graph

Test

Yes - we do have a test process!

Every few hours (at least daily) we checkout all the work in progress svn toolchain, kernel and U-Boot, and run the following tests (over JTAG, serial, and ethernet)

  • binutils - 35
  • gas - 120
  • gdb - 9781
  • ld - 66
  • newlib - 22
  • libstdc++ - 2505
  • gcc - 37612
  • g++ - 12226
  • kernel (LTP) 3000

For details check out the specific testing process for each project.

Asking a Question

The http://blackfin.uclinux.org website is supported by both an active open source community as well as a small dedicated team from Analog Devices, Inc. If you ask a question, it will be answered (normally within 24 hours).

We provide support for the projects via the Analog Devices Engineer Zone forums:

We do not provide generic Blackfin processor support. This is what Analog Devices does. Please ask them. If you have a question about:

  • wiggling bits in registers and seeing xxx results on I/O
  • memory latency
  • electrical specifications
  • peripheral configuration

We do not have the right people monitoring these forums to answer these types of questions. If the question is one where the toolchain, or kernel, or OS doesn't matter (like the above examples), it is likely that you will be asked to ask the question somewhere else (where there are the right people watching/answering questions).

Asking on the wrong forum, is just annoying to project developers, and will delay the appropriate response

If you have a question, about how something works or why it doesn't work, or think you have found a defect/bug, please ask in the Help forum or file a bug report to the bugs tracker linked on the front page.

If you have a question - “How do I do …” - search this on the wiki for the topic, or if there are no topics in the wiki to help (or you can't understand the instructions), please read How to ask a question the smart way by Eric Raymond, and then ask on the help forums. If English is not your first language - that is OK, we can try to answer questions in Chinese, German or English.

Requests reported via telephone or e-mail cannot be properly tracked and managed and so, to help us to support you, please report all issues you have via the forums.

Sometimes when we are trying to answer the question, there isn't enough information in the original message, so we will ask for more information. Sometimes we will ask you to load things up in a debugger. If our answer doesn't help, or you think it is bad advice, tell us, and we will try to determine what your problem is, and a solution for it. If we can't help - we will tell you. We may not have all the answers right away, but we do try to get back to you. If we answer, and you don't follow up, we consider the issue closed.

We understand that all questions are urgent, and that you really need an answer right away. However, asking the same question in multiple forums on the same day, is annoying, and it is likely the questions will be ignored. Also exclamation points ! in subject lines typically do not get read. Every question is answered as soon as possible.

Graph

When asking questions on the forums:

  • New issue -- new thread - it makes it easier to look back on things and determine issues.
  • Don't post to a old message - it is likely that the problem you are suffering from come from something different - even if you are seeing similar symptoms.

How to get a quick reply

The more specific you can be when you ask a question, the more likely we can help you without asking further questions. Be as specific as possible, and when appropriate, include the following information:

  • toolchain release version
  • u-boot release version
  • kernel release version
  • your hardware name, processor type (BF533, BF537, …), chip version
  • daughter board name if any
  • detailed description of the issue
  • for application questions:
    • binary format: FDPIC static, FDPIC shared, FLAT static, FLAT shared
    • command line used to compile
  • for kernel questions:
    • kernel configuration difference vs the default in the release
    • your user/vendor configuration difference vs the default in the release
  • complete error log
  • expected result or behavior if any

Finishing your query with “Please send your reply to… ” makes it unlikely you will get an answer. If you want an e-mail copy when somebody replies in the thread, request that the Web forum send it; this feature is supported if you click “Monitor this forum”.

Getting Help for the Hardware

Analog Devices can be contacted in the following ways (NOTE: Analog Devices can only answer hardware questions about the Blackfin processor. They cannot answer software questions about uClinux or any other program found on the web site):

Analog Devices, Inc.
One Technology Way
P.O. Box 9106
Norwood, MA 02062-9106
USA

Reporting a Bug

Before reporting a bug, please read How to Report Bugs Effectively by Simon Tatham. It is available in over 10 languages. We normally accept bug reports on trunk, and the latest release. If we can not replicate the bug on trunk or the latest release, we will not look at it.

The main purpose of a bug report is to enable us to fix the bug. The most important prerequisite for this is that the report must be complete and self-contained. Before you report a bug, please check:

  1. if there is an existing bug about the same/similar issue.
  2. try a current development snapshot
  3. Before reporting that GCC compiles your code incorrectly, please compile it with -Wall (all warnings turned on) and see whether this shows anything wrong with your code that could be the cause instead of a bug in GCC.

We prefer bug reports in English, but can accept bug reports in Chinese and German, too.

If you want to make a bug reports:

After a bug is reported, it will be validated, and assigned to a developer, and the status will be set to “Assigned (Not Started)”. As the status implies, replication does not mean the developer has started working on the bug yet. Once the developer has time to look at the bug, they will change the status to “Under Debugging”, this means the developer is actively working on this bug. Once the developer has solved the bug, they will commit a fix to the source tree and update the tracker, and set the status to “fixed”, and wait for you to tell us it is solved to your satisfaction. If it was not fixed then let us know, and we will continue to work on the problem. We depend on you for timely feedback. If we are unable to reproduce the issue, and you don't provide any feedback - we will have no choice except to close the bug and mark it as “rejected”.

The easier you make it for us to find/fix/patch a bug, the faster it will get looked into, and fixed. The best thing to do is attach a well thought out, upstream acceptable patch. If this is not possible, attach a simple test case allowing us to replicate your bug. If you can't make a test suite, provide detailed steps to replicate your issue. Without these things, it becomes difficult to root cause a problem. If we don't know what is wrong, we can't fix it. Before reporting your first bug, read How to Report Bugs Effectively by Simon Tatham.

Bug reported via telephone or e-mail cannot be properly tracked, assigned or fixed. To help us to support you, please report all issues you have via the bug tracker.

Graph

We do it this way so we can quickly see: - how many bugs are open - which bugs are being worked on - which bugs have been assigned to someone that has been sitting a long time.

If a bug has been assigned to someone, and they have not started working on it yet, any developer is free to assign that bug to themselves, and start working on it.

Bug Priority

Bugs are prioritized from 1 (most severe) to 5 (least severe).

  • 1 == critical; should be fixed asap
  • 2 == important; fixed before next release, releases are delayed so bugs are fixed
  • 3 == normal; should be fixed before next release, but may slip based on relative priorities
  • 4 == minor; if people find time for it
  • 5 == wish list

Submitting a Patch

If you fixed a bug, you can submit the bug report, and the fix on the bug trackers mentioned above. We only review patches that can be applied to the svn with the patch command (this means can be applied to svn trunk with zero changes - if hunks fail, the review stops there). If you are not familiar with patches, please review patches and diffs.

If you have a feature enhancement, or want to send your platform information to us, attach it to the patch tracker.

It will then get assigned, reviewed, commented on, and either committed or rejected. We can't accept all patches, but we review all patches. We also normally do not accept patches that are not acceptable upstream - for different projects, this means different things. For the toolchain, that means all copyright must be assigned to the Free Software Foundation. For the kernel, ffmpeg, and many other projects, that means meeting a certain coding style. Unfortunately, we do not have the time nor personnel to fix random patches so they are acceptable upstream, so we normally ask the submitter (you) to do it.

For the kernel, it means:

  1. make sure you can run ./scripts/checkpatch on things with no errors or warnings.
  2. for things that do not effect the ./arch/blackfin directory, or Analog Devices device drivers. - you do not need to send to us, these can go directly upstream (the linux kernel mailing list or lkml) -- see ./Documentation/development-process for details.

Graph

Third Party platforms

Often times 3rd parties create their own board and wish to get integrated with the mainline Blackfin tree. In general, we encourage everyone who creates a Blackfin-based board to submit their board resources for inclusion. Doing so benefits everyone involved. For example, when a core change happens that affects boards, we will make the update for you. Having a diverse set of boards also allows us to see the sort of things that people need to accomplish and unify common functions or better architect things to make your life easier. How you specifically get things merged depends on the project.

For the uClinux distribution, we usually just give people commit access to the uclinux-dist project. This way you can manage your vendors directory however you see fit (see vendors for more information). For anything outside of the vendors directory, you should generally go through the normal process of submitting a tracker item, which is submitting something to the uClinux-dist patch tracker.

For the Linux kernel, you should either email your patch to our Blackfin kernel development mailing list or submit to the patch tracker. There you will get feedback on your changes and eventually be merged. See the Linux kernel porting guide for more information.

For the Das U-Boot bootloader, you should email your patch to the Blackfin U-Boot development mailing list or submit to the U-Boot patch tracker. There you will get feedback on your changes and eventually be merged. See the U-Boot porting guide for more information.

Roadmap

The Linux kernel is not developed with big design documents, feature requests and so on. It evolves over time based on the needs at the moment. When it first started out, it only supported one type of processor, as that's all it needed to. Later, a second architecture was added, and then more and more as time went on. And each time a new architecture was added, the developers figured out only what was needed to support that specific architecture, and did the work for that. They didn't do the work in the very beginning to allow for the incredible flexibility of different processor types that we have now, as they didn't know what was going to be needed.

The kernel only changes when it needs to, in ways that it needs to change. It has been scaled down to tiny little processors when that need came about, and was scaled way up when other people wanted to do that. And every time that happened, the code was merged back into the tree to let everyone else benefit from the changes, as that's the license that the kernel is released under.

www.kroah.com_log_images_ols_2006_keynote_08.jpg From Greg Kroah-Hartman's OLS 2006 Keynote.

As for the Blackfin side of the Linux kernel - we do have a little more (but not much) of a process. We do listen to Kernel feature requests, Toolchain feature requests and Das U-Boot feature requests and even implement some of them. We don't implement all requested features - things that are not within our control, have no broad appeal, will not be acceptable upstream, or ideas that we think will be rejected. That doesn't mean that you shouldn't ask, but the answer may be No. This also does not mean that it is not possible, nor does it mean we would not like to see it happen. Since you have the source - you are free to make the modifications with or without our support or help. After you are done (if you decide to do it yourself, send us a patch, and we will review it).

If you submit a feature request, normally, clarification will be requested (we need to understand the detail of what you are looking for, so we don't spend time working on things that are not valuable), please answer in a timely manner, or the feature request will be closed.

For example, things we have rejected in the past include:

  • VDSP++ compatibility - We can change every architecture that gcc supports (including x86) to be compatible with VDSP++ (which we are not going to do), or VDSP++ can become compliant with the industry standards (Since we don't have access to the VDSP++ source (or want it), we can not work on this). If VDSP++ becomes more compatible with gcc or Linux in the future - that is great - but we will still use gcc for all of our projects - and therefore can't provide help or support.
  • glibc compatibility - There are many GNU extensions to the POSIX specification which have crept into glibc - people should not be using these in embedded applications which are targeted for many processors. This is the message that we have received (loud and clear) from the uClibc developers and maintainers - so we pass this advice along to others

Normally every 6 months, we review the feature requests, and either decide:

  • wait (we would like to do it, but it is not high enough priority)
  • close it (for any of the reasons above - we will always comment on why something was closed)
  • plan do do it during the next release cycle.

After the code has been checked into svn, the developer should let whomever is monitoring the feature request know, that this is complete, and they can test it out on svn trunk. After it is decided everything is working, it will be part of future releases. (This is also the reason why we don't add everything under the sun - we maintain and test all features on new releases, and adding too many random things causes tests to be unstable, or take too long).

Graph

Development

Development of any distribution can be quite a complex issue. We manage various projects within the dist, and keep in sync with many outside. Our goal is that any local changes we make are sent upstream to the mainline projects - but this can be pretty large task with the amount of projects that are inside the uClinux distribution now.

Graph

The Linux kernel is a integral part of this, but is handled in a more structured manner, due to it's development process.

Graph

For example;

  • when we make a change to something in linux-2.6.x/arch/blackfin, we commit it to the local blackfin.uclinux.org svn, and the Blackfin git tree, and during the next merge window, Linus will pull from that, and take all our patches/fixes/updates into his tree.
  • when we make a change to something in linux-2.6.x/drivers/serial/bf5xx.c, we commit it to the local blackfin.uclinux.org svn, and send the patch to the serial driver maintainer. This is the only person that Linus will take serial driver patches from. It is up to the serial driver maintainer to take our patches, apply it to their tree, and then push the changes to Linus during the next merge window.
  • when we make a change to common code in linux-2.6.x/kernel, we commit it to the local blackfin.uclinux.org svn, and send the patches to the appropriate maintainer, plus Andrew Morton, and Andrew will add it to the -mm tree (which is not a tree, but a collection of patches). During the next merge window, Linus will pull from either Andrew's tree, or from the subsystem maintainer.
  • when a new kernel is released, we will update the local blackfin.uclinux.org svn, and apply patches from the stable team.

This means that the trees on kernel.org and the ones in the local blackfin.uclinux.org svn will never be exactly the same. The local blackfin.uclinux.org svn will be ahead of the mainline kernel for Blackfin changes and fixes, and behind in terms of general kernel release.

U-Boot is managed in a similar way, where we send patches to the mainline maintainers, and periodically update the local svn to become synced with the mainline project.

Graph

and the Blackfin/GNU Toolchain, which is managed in a similar way, except for the bits and pieces (like the eclipse plugins, and ldr utils, which we create, and maintain 100%):

Graph

While the desire is that people should be able to take a mainline project and have it function, and work with it, this is not the most important thing when it comes to making releases, or doing development. While we spend best efforts in working with the mainline projects, people who develop products with the Blackfin are recommended to get their source code from this site, which has been tested and released for production development.

We also do not always release the most up to date versions of things, and updated projects based on functionality and schedule. If a new kernel or toolchain, or U-Boot release comes out that does not add functionality that we need - we will not upgrade.

Commits

When making an actual commit to one of our repositories, here are some general guidelines to keep in mind. These keep the source maintainable and reviewable, and make merging upstream a lot easier. By and large, these are not suggestions.

  • Follow the coding style of the file/project you are currently working on. This includes, but is not limited to, indentation style, variable naming, and brace placement.
  • Do not leave trailing whitespace; either at a specific line, or at the end of a file.
  • Do not mix style changes with functional changes. If you need to fix white space (incorrect/broken indentation, trailing space, etc…), then do this in a commit by itself.
  • Do not mix unrelated changes. This is similar to the previous item. If you fix multiple bugs, then you should generally make multiple commits. You should certainly never mix new features/functionality with bug fixes in the same commit.
  • Follow the GIT commit style. A general template:
    [#trackerid] subproject: area: one-liner summary
    
    Following a blank line, write out an explanation of what this commit is for.
    It must be complete enough for people to understand what is going on without
    having to look up bug reports or read mailing lists.  Referencing such things
    is OK, but they should not be a requirement.  Simple messages like "fix bug"
    or "board is broken" is not acceptable with the upstream projects, thus it is
    not acceptable in our repository.  Every time a short commit is made like that,
    you force a lot of unnecessary busy work on the person responsible for bundling
    and sending your changes upstream.
    
    Also, feel free to credit people, even if they only reported the issue.

Some notes specific to some projects:

  • linux-kernel
    • Split out changes to architecture-specific files from common files. Upstream will not accept a change that modifies a driver (like the Blackfin NAND driver in drivers/mtd/nand/) and modifies architecture specific pieces (like header files in arch/blackfin/include/asm/). So you will need to split this.
    • Split out updates to boards files (arch/blackfin/mach-*/boards/*) from other files. Similar to the previous item, you should make commits to boards files (like arch/blackfin/mach-bf537/boards/stamp.c) separate from the related drivers (like sound/soc/codecs/).

Releases

When we do a major release, the first thing to do is understand the names. For example 2008R1-RC5, is the fifth version (or release candidate -RC5) of the first release (R1) in 2008 (We are not very innovative when it comes to names). We do not have “Golden” releases, since every time we have made one in the past, we end up finding/fixing more bugs, and have further releases after the final one - which was confusing to developers and users.

The difference between the different release candidates are simple (but sometimes major) bug fixes. Since the bug tracking/fixing/releasing cycle may be longer/shorter for the three main projects (toolchain, Das U-Boot, and uClinux-dist (including kernel)), it should be expected that the release candidate numbers for different projects will be different.

When we discuss/make a release, we normally talk about things as “2008R1”, and do not include the release candidate suffix - after all, you should be using the latest version, which will be different between the toolchain, uClinux-dist/kernel, and U-Boot. Which is the latest, latest version? SVN is always the master. However, svn can appear to be a complex labyrinth of trunk, branches and tags. This describes how svn is managed:

  • All development happens on trunk, this includes upgrades of packages to current mainline.
  • Branches are used for releases, which will be modified over time, to include more bug fixes. Normally, new features are not included in a branch, but we have been known to backport minor features, which do not effect binary compatibility. Upgrades of packages in a branch is forbidden.
  • Tags are snapshots in time of branches, and are used to mark what svn looked like when we posted files on the release section of the web site. This allows you to recreate the tarballs if you want.

For example:

Graph

This describes the typical development:

  • Development, adding of new features, updates to main line projects happens in svn trunk.
  • When things are close to being stable, and most planned features are in trunk, we make a release branch. tag it with a -RC1 (release candidate 1), and start testing it.
  • If/When we find a bug, and fix it (bug fix1), we apply the fix to the release branch and trunk.
  • Depending on timing, severity of the bug, we will either wait to find more bugs, or tag the branch with -RC2 (release candidate 2), and re-start testing.
  • Their may be times where we find bugs in -RC2, but the severity of the bug is low, and we will not make a -RC3. For those people who want the bug fix, they can go to the branch, and get the latest and greatest.
  • New features will be added to trunk, and we may uncover new bugs that don't exist in the branch. These will get fixed only on trunk.
  • If bug exists in the branch that can be validated on other architectures (not a Blackfin specific issue), we may not fix it in the branch. (For example - if there was an architecture issue in a piece of software, and the “bug” was fixed up upgrading to the latest mainline code - we will not do that to the branch, because of the other instabilities that the upgrade could cause).
  • eventually, there will be enough new features, and new upgrades in the trunk that we need to make a new release. There are no hard schedules. If you must have a specific feature in a specific release, please contact us.

What this means:

  • The releases we have on the release pages of the web site should correspond exactly to the tags in svn.
  • Use the highest release candidate -RCn that you can find on the web site.
  • For toolchain and U-Boot - these are very stable, and we do not (normally) fix things in the branch, that are not posted.
  • There will be bug fixes in the svn branches that are not in the release files of the web site. For 99% of developers this should be OK.

To learn more about svn, and checking out from a branch, check out blackfin_live_sources.

Supporting Releases

We support (answer questions, fix bugs, etc) on the latest release. In the above diagram - that means that after 2009R1 branch is made - we normally stop applying bugs fixes to the older branches - since we expect people to keep up with our developments.

Trunk may not always be stable (but we try), and some features may be broken on trunk (sorry - they normally get fixed soon). Developing off the latest release branch is always the best idea.

Web Site

As you may have noticed - all of our development activity revolves around the web site. It is the center of our world (but we don't get out much). All questions, support, documentation, source code, bugs, patches, feature requests, and planning - all happen on the web site. The more you get comfortable using it, the better experience you will have with it.

Graph

Complete Table of Contents/Topics