Building Real-Time, Multi-Tasking Embedded Applications Without an OS Kernel

Building Real-Time, Multi-Tasking Embedded Applications Without an OS Kernel

pseudOS™

Building real-time, multi-tasking embedded applications without an OS kernel.

Bruce Boatner, EE

Preface

Your project starts off well enough – a simple and elegant solution to a well-defined set of features and functions. You are half-way to your first test release when the changes start coming in. That important customer who’s promised to buy thousands of your new product has started thinking about all the new features that “would be nice to have”, and your company’s sales guy, not wanting to risk losing the big deal, has promised them the moon. “Sure – our guys are real smart – they can do anything!” we says, with a grin.

Suddenly things don’t look so pretty. Your design is now encumbered with all kinds of timing issues, unwieldy inter-dependencies and pretzel logic, as you valiantly attempt to incorporate the new functionality into the program while still meeting the required system response times and project time lines. Your “simple and elegant” solution is showing the potential of turning into a nightmare.

You remember someone once having said “I never could have done that without an RTOS”. Even if you could get your current system to work, you know the person who will eventually end up maintaining it will be cursing you under his or her breath, knowing that any change they make might wind up producing a myriad of unintended consequences. That’s not exactly the legacy you hoped to leave in your wake.

You are now faced with the unpleasant prospect of researching, identifying and acquiring an appropriate RTOS, and re-architecting the entire project in the OS environment. As if you needed more pressure, you are aware that many projects and careers have not survived the schedule and budget impacts of such a major course correction.

Inevitably, then, these situations seem to come down to a choice between designing a standard, single-threaded, relatively simple architecture, or using a multi-tasking RTOS. There appear to be no other options, with the “No Man’s Land” in between the two choices being fraught with spaghetti code monsters and un-decipherable C creatures.

But what if this were not the case? What if there was a third choice?

pseudOS is designed to fit into this gap. It is not intended to replace the RTOS, but rather to emulate many of its behaviors. There are times when using an RTOS is absolutely the correct choice, especially in complex systems involving the coordinated efforts of multiple developers, but many times the majority of the features of an RTOS are not used. Closer analysis shows that simplemulti-tasking functionality formsthe basis for choosing an RTOS the majority of the time.

pseudOS is intended for programmers who want another choice, or who are starting to lose control of their applications due to the reasons described above. Since pseudOSis a design methodology, any existing single-threaded application can be “retro-fitted” to the multi-threaded pseudOSphilosophy by following a simple set of guidelines. This usually involves breaking down existing functions into smaller pieces, organizing them into groups of actions that support state functionality, and executing these action-objects under a finite state machine control system. The result is a system that is logical, easy to maintain, and highly responsive.

Introduction

The idea of pseudOScame while exploring the possibilities of using large numbers of inexpensive micro-controllers to emulate neurons in Artificial Neural Network (ANN) designs. Generic neurons take input signals from other neurons, and based on this activity and asystem of weighted values,produce an appropriate output. The “intelligence”, if you can call it that, comes from the configuration of the weights at its inputs.

Since most micro-controllers, or Systems on Chip (SoC) can typically manage a large number of analog inputs and have both digital and PWM outputs (PWM is easily be converted to analog voltage for inputs to other neurons), it seemed like a natural fit. A simple bus – I2C or SPI - could be used to train the system and establish the weights. Of course it would be desirable to pack as many neuron emulators as possible onto a single SOC part, so an RTOS appeared to bea logical part of the equation.

One interesting thing about neural networks is that they are not designed to produce a pre-determined set of outputs from a known set of inputs – they are designed to learn to produce appropriate behavior through training, with a healthy dose of noise and uncertainty thrown in for good measure.

Another interesting feature of neurons and neuron clusters, is that they tend to do one thing very well, like recognizing a pattern, and actually have no clue what the overall goal of the system is, or how the results of their efforts are being used. All they know is that they get an “atta boy” when they do things right, and that’s good enough for them. It is the system, at some higher level, that sends out the “atta-boys” when the organism has found something yummy to eat, or escaped from something that considered itsomething yummy to eat.

The organism is state-oriented – it may be in a state of hunger, or a state of panic. It is also interrupt and priority-driven. It might be configuredin an optimal state for finding food, when it is interrupted by something appearing on the scene which causes it to enter a state of panic, the priority being survival. At that point all neural activity is switched to the necessities of flight.

Let’s examine how a state change might occur in the organism. As we implied before, the individual neurons, neuron clusters and processing sub-systems are organized in a hierarchical manner. Their “raisond'être” is to provide accurate information to the next level higher-upin an efficient and timely manner (the right answer late is the wrong answer). Their reward for this is an “atta-boy”, or not getting eaten. Perhaps if things are not done as well as expected, there might be a mild electric shock involved.

So let’s look at this massive array of neural clusters that is at the disposal of the organism at any given moment. Neurons are cheap; when the organism is growing, it produces them by the billions. If it is a matter of survival to be able to switch states instantly, would it make sense when so much is at stake to depend on the same set of neural that were specialized in sniffing out food to suddenly become experts at survival tactics?

Probably not. Just as we have soldiers, dentists and plumbers that are trained and equipped to support specific needs of our society, it would make more sense that specialized neural configurations could be switched in and out quickly when a significant state change is called for. Remember, neurons are cheap and there are lots of them. And you wouldn’t call a plumber when you have a tooth ache, or muster the dentists if we just got invaded by Uruguay.

In the pseudOSthese concepts have direct corollaries. Functions are equivalent to neurons (neuron clusters, actually). They are small, simple and efficient – and there are lots of them. Functions are considered “action-objects”. Like neurons, they respond to the outputs of other functions and they also depend on their own internal state (as a neuron maintains its weights). When active, they are cycled repeatedly and frequently. Many times they may do nothing more than checking an input bit and exiting. When they do something, it may be as simple as setting a flag, putting a character in a buffer, or incrementing a global variable.

Like the organic model, pseudOS is state oriented. States are created by switching on and off setsof “action-objects” to produce the desired behavior. Near parallel-processing responses can be produced by running large numbers of small “action-objects”as quickly as possible. The “action-objects” are not run continuously, but a cyclical or pulsed manner. This establishes a low duty-cycle where the processor is free to respond to non-deterministic events and interrupts.

Studies have shown that the brain processes data in a cyclical manner, which can cause adverse responses from strobe lights and other stimuli that closely match the natural rate. Without a “system clock”, randomly wired parallel systems can become a non-deterministic free-for-all, where variable signal path lengths cause race conditions, instability and inevitable chaos.

The cyclical nature of organic computational systems is not the same as the “system clock” of a man-made state machine, however. It is more like a wave-front traveling through the computational clusters, orchestrating hierarchical processing by controlling the order of sequential sub-functions. Thus it is an interesting mix of parallel processing accomplished by sequential execution. It is this process that we attempt to emulate in pseudOS applications.

Before delving deeper into what pseudOS is (and isn’t), let’s take a quick look at some of the RTOS varieties you might encounter in your research.

Proprietary RTOS

These products range from mission-critical applications like aircraft and engine control systems, medical equipment, etc., to small embedded systems that might manage a power tool or microwave oven. Most of these products, since they tend towards being rather expensive, come fully featured with extensive API’s (Application Program Interfaces). You can tell something about the complexity of these offerings by the length of their user manuals, which are typically many hundreds of pages in length. “Creeping Featurism” is common, and the size and scope of these products is impressive, the cost of which is reflected not only in the initial ‘seat’ purchase price, but also complex distribution licensing fee structures. Although its size can be regulated by loading in only the features that are required for the application, the kernel size of OS is also an important constraint on severely resource-limited processors.

However, one must also take into consideration “you get what you pay for”. Purchasers of proprietary software packages are typically provided excellent customer service access, the value of which cannot be underestimated in a time-critical development process. The seasoned engineering manager could certainly be considered diligent for concluding that the success of a project is worth the insurance of having a professional available by phone. Overcoming a single “show stopper” in a timely manner can easily justify the ticket price.

Open Source RTOS

The GPL (General Public License), through the Free Software Foundation, is becoming a popular means of acquiring and distributing software. Linux is perhaps the “poster child” of the GPL model, and we will discuss embedded Linux in a moment.

Two examples of small-footprint RTOS systems are freeRTOS and PICos18, which can run on micro-controllers with limited resources. This is also an ideal environment for pseudOS, since any RTOS imposes significant resource requirements before the first line of application code is ever written.

Perhaps the most significant example of a resource-hungry RTOS is embedded Linux. The original multi-process Linux OS incorporated over 125 million lines of code, with some “skinny” versions touting a “mere” 45 million LOC. It is not unusual for a minimal implementation of an embedded Linux system to host a kernel of 5-6 KB. Embedded Linux is a streamlined version, but it is still Linux, and carries with it all of the characteristics of its big brothers, including the need to mount a disk file system, program paging, boot-loaders, run-time code image expansion, etc. Embedded Linux is typically hosted on very powerful micro-controllers and processors, incorporating Memory Management Units (MMUs) to accommodate the heavy memory usage requirements, and very high clock speeds necessary to accommodate the hundreds or thousands of cycles necessary to perform a simple context switch.

Linux, like Unix, is a powerful multi-user, multi-processor, multi-tasking OS, with every bell and whistle known to man. It is ideal for the engineering program development environment, but why in the world would anyone in their right mind consider it as a solution to an embedded problem?

First, Linux is “free”. However, the concept of open source software contains some subtle and potentially insidious implications to the commercial product developer. You will hear the term “GPL infection”, meaning the potential for incorporating GPL code mixed with proprietary code requiring the developer to “open source” program materials containing trade secrets and other intellectual property. Simply observing the extent to which discussions of how incorporating GPL software affects or does not affect commercial enterprise and intellectual property says a great about how uncertain the legal precedents are in this area. My question is – if you could avoid this conundrum altogether, why would you ever bother with it in the first place?

Secondly, there is an understandable desire in the industry to converge on a standard in the world of embedded devices. I am certainly sympathetic to this idea as a response to the bewildering array of closed-source proprietary solutions and tool chains, but unfortunately I do not believe that Linux is the answer. It simply appears to have been the only logical choice available at a time when the engineering community was looking for alternatives.

What is pseudOS?

“pseudOS is a design methodologythat emulates multi-tasking functionality at the application level.”

Let’s break down this description down into a high-level overview.

1. pseudOS is a design philosophy. Specifically it exploitsfinite state machineconcepts to provide multi-tasking functionality, usingstandard ANSI C programming practices.

2. pseudOS provides functionality at the application level. An RTOS kernel controls the execution of program code based on priorities, events and resource availabilities, whereas pseudOS provides a basic-level multi-tasking functionality, upon which additional control features of the RTOS can be emulated as required.

3. pseudOS is a tool used to produce applications that meet prescribed program specifications regarding behavior patterns and response times. In a black-box analysis, emphasis is placed on what the application does, not how it does it.

Guidelines for pseudOS design

1. Functions are structured to perform single, well-defined “actions”. They may be regarded as “action-objects”, and typically contain local state using static variables. Taking this approach means there will be a larger number of functions that may be typical in the standard application. Although it is tempting to consider these “action-object” functions as tasks, this terminology is avoided to reduce confusion. Also, there is no real attempt to limit the number of “actions” in pseudOS, whereas in the RTOS environment there is typically an upper limit to the number of tasks that can be reasonably managed.

2. pseudOS is state-oriented, and applications are structured using a finite state machine approach. If you have not used FSM techniques, you will find that the ideas are very straightforward and easy to grasp. Not only are FSM design concepts intuitive, but it is the high level of determinism – never having to guess what state you are in, and what state you will be in next based on certain events – that make the methodology one of the most attractive and powerful tools available to the designer.

3. pseudOS states are action-based. A state is determined by what behaviors the system will exhibit, and these behaviors are controlled by the “actions” designated for that state. For each state, an “action list” contains the “actions” that will be executed to produce its desired functions and responses. The “action” functions are executed in continuous round-robin fashion until the state changes and a new “action list” becomes active.

4. State changes are initiated by events processed by the “action” functions. pseudOS is not a pre-emptive system, meaning code execution is not scheduled by an outside resource (RTOS kernel) because there is not one present in the system. In this regard it is closer to a “cooperative” approach in which a function (task) can defer functionality to another function (task). However, this is a misleading analogy since the “cooperative” philosophy is also an RTOS concept, meaning it requires a scheduler which is part of an OS kernel.

5. The basic functional mode of pseudOSis an equal priority, round-robin execution process. This provides the basic multi-tasking capabilities that are frequently adequate to meet the requirements of an RTOS-like application. There is still a great deal of flexibility even within the perceived limitations of the RR approach, using flags and inter-action semaphores, system variables, counters and timers, interrupt service routines, action list execution orders, etc. If behavior specifically related to priorities or resource management is required, these can be emulated in a number of ways “on top of” the RR approach. Even in the minimal RTOS implementation, a set of baseline functionality is present, whether or not it is utilized. In pseudOS, functional code is introduced only as needed, maximizing the usage of scarce resources.