Decisions Behind the Design: LabVIEW for CompactRIO Sample...
40
ni.com Decisions Behind the Design: LabVIEW for CompactRIO Sample Projects
Decisions Behind the Design: LabVIEW for CompactRIO Sample ...australia.ni.com/sites/default/files/Decisions Behind the Design... · ni.com Decisions Behind the Design: LabVIEW for
Decisions Behind the Design: LabVIEW for CompactRIO
Sample Projects
2 ni.com
Agenda
• Keys to quality in a software architecture • Software architecture overview • I/O safe states • Watchdog timers • Message communication • Error handling • System monitoring
3 ni.com
Keys to Quality in a Software Architecture
• Define software architecture and identify architecture components
• Create an architectural diagram • Consider key components such as data communication,
error handling, etc. • Learn the foundational design patterns • Create new design patterns as required by your
application (requires experience)
Presenter
Presentation Notes
-Question for the audience: have you ever designed an application for a cRIO/sbRIO/R series system? (if no one raises their hand, walk them through this slide). If they have, ask them what techniques they applied that were different than designing a Windows based application. Ask them what techniques were the same. In our experience, we’ve noticed that it’s more important for RIO applications to create a design diagram; that’s because you have so many programming targets, it can be difficult to keep track of everything. These diagrams include all of the system processes, and also where they are located – on the FPGA, RT host, or HMI. We’ve also found that data communication tends to be more prevalent/important in embedded applications, again, because you have three targets to communicate between, in addition to the overall systems being more distributed (often a lot of networking). In the design diagram, it’s important to document the data communication paths in addition to the processes or loops. Error handling is also critical – since many embedded systems are typically running headless, untouched for days, weeks, even years, it’s important for these systems to both communicate any errors up to an HMI, and also to resolve errors while the app is running, to avoid having to shut down the system and perform maintenance. Some areas where you can leverage your windows development is when it comes to design patterns. State machines, queued message handlers, etc. work great on embedded systems – a state machine can be implemented in LV RT or LV FPGA. Today we will look at an example of an app that uses a state machine on FPGA. The queued message handler framework is great for running on RT and distributing messages between the HMI and FPGA target – also for handling errors. You can also create new design patterns, versus reusing the ones that NI provides, for your applications. This requires some experience, but will ultimately save you a lot of time. We have some design patterns that our Systems Engineers have created, available at ni.com/referencedeigns. The “Reuse Libraries Everyone Should Know” session later this day, will go more into detail on some of these available design patterns and APIs.
4 ni.com
Data Communication Diagram
• Documents foundational components: • Processes (loops) • Data communication paths • Type of data transfer
Presenter
Presentation Notes
There are several different approaches to diagramming a system – the type of diagram that we recommend for RIO applications is a data communication diagram, similar to the one shown here. These diagrams document the different processes or loops, and the data communication paths. In addition, it is very important to understand the type of data communication, because each type of data communication has different attributes, and LabVIEW offers many APIs that satisfy each type of communication – some are better at communication certain types of data than others. Also, the type of data communication might influence how you design your processes.
5 ni.com
Data Communication Types
Tag Latest value data
Messages/Commands Intermittent data, low latency
Stream Continuous acquisition, high throughput
Presenter
Presentation Notes
The three most common types of data communication that we see within embedded systems are listed here. A tag is current value data that we are writing to or reading from periodically. An example of a tag communication is monitoring the current value of I/O channels, such as the current temperature. We don’t need a buffer because we are only concerned about the latest value. An example of a tag that we would write to is a setpoint. In order for a PID control loop to function correctly, we need to be monitoring the setpoint within our loop on every iteration. Tags are often implemented with variables. A message or command is usually generated from a user interface, but not always. Generally, commands and messages are sent intermittently, and require low latency. For example, if the command is “emergency stop”, we probably couldn’t tolerate any sort of delay in our communication. Commands also require a small buffer to ensure that no command is lost. Also, as soon as a command is processed, it should be removed from the buffer so that it is only read once. Examples of commands are “Start Acquisition”, “Log Data”, etc. Tags and messages are inherently very different, and often times messages are implemented as tags which can cause performance and reliability issues. Streaming is pretty straightforward – you’re communicating a large amount of data with very high throughput. An example is a waveform or image.
6 ni.com
Typical CompactRIO System Diagram
Presenter
Presentation Notes
This is a diagram of a typical embedded control system implemented on a cRIO device, which leverages the FPGA for control and reliability. The black arrows represent the transfer of data, and the red arrows represent the transfer of error messages. Talking points: FPGA based control FPGA based watchdog to monitor RT health Two network paths, one for messages, one for tags Easy to separate communication from control Easy simulation of the RIO device or HMI
7 ni.com
LabVIEW for CompactRIO Sample Projects
• Pre-built architectures for embedded control and monitoring applications
• Designed to ensure quality and scalability of a system
Presenter
Presentation Notes
The architecture that you just saw is actually shipping in LabVIEW 2012 and later as a LabVIEW for CompactRIO Sample Project. We are providing this software architecture, in addition to four others: -LV FPGA Waverform acq & Data Logging -LV RT Control (RIO Scan Interface) -LV RT Sequencer and Control (2013 and later) -LV Datalogging and Supervisory control (2013 and later) In this session, we are going to walk though the LabVIEW FPGA Control Sample Project architecture. This architecture is very similar to the other sample projects, so if you can understand the design that we used here, you should have a easier time getting started with the other sample projects as well.
8 ni.com
Basic FPGA Diagram
Performs deterministic and/or high speed control
Presenter
Presentation Notes
Starting with the FPGA control loop…here we are performing closed loop PID control with analog input and output signals. We are implementing the control loop on the FPGA in order to achieve deterministic control with high loop rates.
9 ni.com
State Machine for Hardware I/O and Control
Presenter
Presentation Notes
Our FPGA control loop in the cRIO Sample Projects is implemented as a state machine. This is recommended for several reasons, one reason being that it helps you create scalable, readable code, and the other, is that you can easily switch between your control state and a safe state. A safe state allows you to put all of your hardware outputs into a safe, known state in case anything goes wrong in your system. This is especially important if your cRIO is connected to expensive, or dangerous machinery. In this state you might want to write “0” to your output values, or, you may want to have some sort of ramp down state versus setting them all to zero immediately.
10 ni.com
Failure Conditions that Initiate Safe States
• Examples (from Fail Safe Reference Design):
• RT Safe – indicates the RT system is ready
• Emergency Safe – tied to an emergency shut-off switch
• Watchdog Safe – monitors the Real-Time system
• Control Inputs Valid – monitors the inputs to the control algorithm
• Based on system requirements
Presenter
Presentation Notes
These are some examples of conditions that you may want to consider when putting your system into a safe state. You may want to check to see if the RT app is up and running, especially after you just rebooted your system, before reconnecting to I/O. You may have an emergency shut-off switch. A watchdog failure is another condition in which you would want to go into a safe state – we will talk more about that next. Finally, if a user provides inputs that will potentially disrupt your control algorithm, you could also go into a safe state. These are just examples – ultimately the decisions you make here should be based on your system requirements. When leaving a safe state, make sure to have our system actively tell you that its okay to leave, you don’t want to just depend on it not throwing an error.
11 ni.com
Basic FPGA Diagram
Detects software failures
Recovers from software failures
Presenter
Presentation Notes
The next process that we will talk about is the watchdog process. Here we are using an FPGA watchdog, which interfaces directly to an RT Watchdog loop.
12 ni.com
Watchdogs
• A watchdog timer is a hardware counter that interfaces with the embedded software application to detect and recover from software failures
• A user can then: • Reboot real-time target automatically • Perform user-defined recovery actions
• Two types of watchdogs with NI Real-Time hardware: • LabVIEW Real-Time Watchdog • Real-Time <-> FPGA Watchdog (FPGA Fail Safe Design)
Presenter
Presentation Notes
A watchdog timer is a hardware counter that interfaces with the embedded software application to detect and recover from software failures. During normal operation, the software application initiates the hardware timer to count down from a specific number at a known increment and defines the action to take if the timer reaches zero. After the application starts the watchdog timer, it periodically resets the timer to ensure that the timer never reaches zero, as shown in the illustration above. If a software failure prevents the application from resetting the timer, the timeout eventually expires because the hardware counter is independent of the software and thus continues to count down until it reaches zero. When the watchdog timer expires, the hardware triggers the recovery procedure. Upon a watchdog failure, a user has the option to reboot the RT target automatically, or perform user-defined recovery actions. There are two types of watchdogs you can implement for cRIO devices – the LabVIEW Real-Time watchdog, and the LabVIEW FPGA watchdog.
13 ni.com
LabVIEW Real-Time Only Watchdog
• Uses hardware timer built into CompactRIO hardware • “Reset = True” reboots system if the watchdog process
is starved
Presenter
Presentation Notes
The LabVIEW Real-Time watchdog interfaces to a watchdog timer hardware component that is included in every cRIO. The LabVIEW Real-Time module provides an API to interface to that watchdog timer, shown above. In this example, we have a timeout set to 1 second, and we are “whacking” or “petting” (depending on how nice you want to be) every 100 milliseconds. This means, that if the RT app hangs or becomes unresponsive for 1 second, we will trigger a watchdog failure and the RTOS will restart. Often times, the RT app will hang because it runs out of memory, and in this case, rebooting the RTOS will free up all memory and the application should be able to run again.
14 ni.com
LabVIEW Real-Time Watchdog
• Enable occurrence in expiration actions • Configure appropriate watchdog timeout and Watchdog
Whack loop period
Presenter
Presentation Notes
You also have the option for the watchdog API to fire an occurrence whenever there is a watchdog failure, so that you can implement your own code to handle the situation. If the RT app was nonresponsive because the CPU was at 100% for example, you could potentially adjust the rate at which some of the loops are running.
15 ni.com
Real-Time <-> FPGA Watchdog Reset timer
Put control loop into a safe state, and reset system
Presenter
Presentation Notes
Another option (and in some cases, a better option), is to implement your own watchdog timer into the FPGA fabric of your cRIO device, using the LabVIEW FPGA module. The biggest benefit here, is that your I/O is also most likely implemented in the FPGA. Therefore, if the RT app becomes unresponsive, you have some control over the I/O before you reset your system. You can put your I/O into a safe state, and THEN restart the RT app. If your application is RT only (no FPGA component), then you will most likely not have control over your I/O if something goes wrong.
16 ni.com
Real-Time <-> FPGA Watchdog
• Pet the watchdog at a user-defined watchdog pet rate • This resets the counter implemented in the FPGA VI
See Fail-Safe Control Reference Design whitepaper
Presenter
Presentation Notes
When implementing a watchdog timer in LabVIEW FPGA, you will develop your own API interface to the watchdog using the FPGA Interface Functions. Instead of writing this yourself, from scratch, you can reference code from our CompactRIO Sample Projects, and also the Fail-Safe Control Reference Design that can be found on ni.com. The Fail Safe Reference design is a reference design pattern created by NI Systems Engineering which provides a starting point for incorporating an FPGA based watchdog and safe states into your embedded system. This design pattern was designed for CompactRIO, but with minor modifications it applies to a PXI RIO, or any FPGA based control application
17 ni.com
Basic FPGA Diagram - Summary
Detects software failures
Recovers from software failures
Performs deterministic and/or high speed control
Presenter
Presentation Notes
Here is a review of the FPGA software components in our system.
18 ni.com
Basic Real-Time Processor Diagram
Periodically communicates tags
Communicates messages with client
Sends messages to other processes or
targets
Presenter
Presentation Notes
Moving onto the real-time processor. On the real-time processor we have quite a lot of message communication. Our message handler loop is receiving incoming messages from the command parser, such as “update PID setpoints” or “exit”. These messages are then being distributed to other loops in the LabVIEW Real-Time application, or sent down to the FPGA. The error messages (or red arrows) are being sent in two directions – they are being logged to the local disk, and they are also being sent back up to the HMI for display. The system monitoring loop is receiving I/O data and error messages from the LabVIEW FPGA application. The error messages are sent to the Message Handler, and the I/O data is being sent directly to the HMI for display.
19 ni.com
Constructing a Message
Examples
Data Variant allows data-type to vary. Different messages may require different data
Command String constant allows user to specify message
Command Data
Update Control Configuration Cluster containing configuration data
Update FPGA State Enum
Update PID Setpoints Array of numerics
Send Error to UI Error cluster
Exit -
Presenter
Presentation Notes
It’s important to consider how you construct your messages, before implementing message communication. The message cluster that you see here is idea for message communication. Within this cluster we have two main elements – the message itself (stored as a string), and any data associated with the message (stored as a variant). By using this construct, you can design a very scalable system. For example, the table here shows several of the commands that are communicated within the LV FPGA Control Sample Project. Each command has a different type of data associated with it. By using the construct shown above, we can send all of these different messages, and more, without changing our messaging framework. (There is a reason why we used strings instead of enums for the messages – for an explanation, see Darren’s “Decisions Behind the Design: Queued Message Handler Template”)
20 ni.com
Queued Message Handler API
Presenter
Presentation Notes
This is the API that we use for passing around messages. It may look new and intimidating at first, but all it is, is a thin wrapper around Queues, which you’re probably familiar with. For example, the Enqueue Message VI contains an Enqueue Element function. All we are doing in addition to this, is bundling the message and message data together into a cluster, and checking whether or not this is a priority message. If it is a priority message, we will use the Enqueue Element at Opposite End function to place the element at the front of the Queue.
21 ni.com
Queued Message Handler Framework
Message handling loop
Watchdog Loop
Monitoring Loop
Command Parser Queued Message Handler Setup
Presenter
Presentation Notes
This is the Queued Message Handler framework that uses the Queued Message Handler API. In this framework, we perform some initialization (set our buffer size to 1000 elements), and then we start our application. You’ll notice that this application here is the RT application that is included with our FPGA Control Sample Project. We have the same four loops that we saw on the data communication diagram: command parser, message handler, watchdog loop, and monitoring loop.
22 ni.com
Queued Message Handler Framework
Watchdog Loop
Monitoring Loop
Command Parser One or more messages handled per case
Communicate messages between processes or targets
Presenter
Presentation Notes
All messages from the UI are coming into the Command Parser loop (over Network Streams), and are being handled in the Message Handler loop. For example, the command “Update FPGA State” is being processed here. The first step is to take the variant data contained within the message and convert it to the enum datatype. Next, we send that command to the FPGA using LV FPGA Interface functions. It’s very simple, but if you try to design an architecture without this extra step in place (for example, sending commands directly from your command parser loop to the FPGA), you will end up with a design that is not scalable, and is difficult to debug. If we decided later on that we want to change out Network Streams and communicate over the network via TCP/IP, we can modify our Network Communication (command parser) loop without changing the code that handles our message communication.
23 ni.com
Basic Real-Time Processor Diagram
Handles all error messages from FPGA
and RT target
Presenter
Presentation Notes
Earlier we talked about the importance of error handling when designing an embedded system. In addition to handling all of our user interface-generated messages, the Message Handler also acts as a central error handler. In addition, each of the other loops that you see here, has its own specific error handler.
24 ni.com
Error Messages
• Specific Error Handling • Code called in specific locations
to respond with an action to specific error codes
• Possible actions are retry, ignore, correct
• Central Error Handling • High-level code that checks for
errors in an entire system • Responds to classes of errors
rather than specific codes • Uses the classification to
determine which actions to take
Queue Central Handler
Presenter
Presentation Notes
Specific error handling and central error handling are two potential implementations for an error handling strategy. The Sample Projects use a combination of both. Specific error handling uses code called in specific locations to respond with an action to specific error codes. Possible actions are retry, ignore, correct. Central error handling is high-level code that checks for errors in an entire system. The central error handler responds to classes of errors rather than specific codes. The classification is used to determine which actions to take. (additional details on central error handling…) Central Error Handling A central error handler is code placed in a central location, such as the end of each main loop, that detects any error and responds to it. Central error handling does not respond to specific error codes, however, it may take different actions based upon the type of the error. For example, a very simple general error handling strategy is to stop a loop in the event of an error. This general error handler does not take different actions for different error codes, but it does divide errors into two types. This general error handling strategy is appropriate for simple programs, however, large programs with multiple loops and more robust interfaces require a more complex general error handling strategy. The easiest way to create a central error handling mechanism is to create a module or subVI to perform error handling. You may also want to convert the subVI to a functional global variable to allow internal data storage and multiple commands. A central error handling module might have the following features: The module should be capable of detecting an error and classifying the error based upon the defined error types The module should be capable of logging errors to disk. Generally, the module should open and close the reference to the error log file each time it needs to add an entry. This ensures that the file is kept in a safe state and not locked. Other programs can view or add to the file while the program is running. If other programs locking the file is a problem or if you are concerned about execution speed, you can maintain an open file reference using the functional global variable. The module should be capable of displaying prompts to the user. For real-time systems The module should initiate a critical system shutdown in the event of a critical error. In some cases, this may be done by storing all of the appropriate references to shut down the system directly within the error handler. In other cases, the error handler may call a more centralized shutdown module to initiate the shutdown. The module should store errors internally. This is less critical if the errors are logged, but for ease of implementation, it is still advisable to keep an internal array of errors. Knowing whether previous errors have occurred is important to avoid initiating multiple shutdown requests, and for reporting errors at the end of execution. The module should clear errors before executing. Warnings and non-critical errors must be cleared so that the program can return to a normal operating state. Critical errors must be cleared so that they do not prevent the shutdown code from executing. In either case, it is safe to clear the error because the error handling module has already reacted to and stored the error. You should call the error handling module at the end of each main loop within your program. This provides good coverage for any errors that might occur. In some cases, you may want to call the error handler from other critical locations, to immediately handle potential areas. Since the error module is a subVI, this is easily done. However, use care when calling the module in time-critical sections of code, as it is a non-reentrant VI, and therefore a shared resource.
25 ni.com
Central Error Handling Framework
UI message handling loop
Watchdog Loop
Monitoring Loop
Command Receiver Loop Specific Handler
Specific Handler
Specific Handler
Presenter
Presentation Notes
Each loop in our LV RT application has its own specific handler. For example, the Command Receiver loop handles our network communication using Network Streams. Network Streams will produce an error message if the client disconnects. In our case, it’s okay if the client disconnects, because our Sample Projects include code that handles that disconnection, so that the user never needs to know about it. Therefore, our specific error handler ignores this specific error. This error is never sent to the central error handler.
26 ni.com
Central Error Handling Framework
Watchdog Loop
Monitoring Loop
Command Receiver Loop Specific Handler
Specific Handler
Specific Handler
Presenter
Presentation Notes
Our Message Handling loop is the central error handler. All error messages, unless they were resolved within the specific handlers, are sent here, along with a classification. The classifications that we use in this sample project are critical, non-critical, and log-only.
27 ni.com
Central Error Handling Framework
Get next error based on priority
Return classification of error
Handle classified errors (update FPGA state and/or reboot system)
Log and send all errors to UI
Presenter
Presentation Notes
When an error arrives, the first thing we do is check the priority. Any priority messages are read off of the front of the queue. Next, we convert the variant datatype that we received into an error cluster type. Next, we log and send ALL error messages to the user interface, regardless of its classification. Finally, we look at the classification, and we perform a different action depending on what the classification is. For example, if the error is classified as non-critical, we put the FPGA VI into a safe state, and we send that mode change to the user interface. If the error is classified as critical, we put the FPGA VI into a safe state and reboot the RT controller.
29 ni.com
Basic Real-Time Processor Diagram
Communicates messages with client
Presenter
Presentation Notes
Next, we will look at the command parser loop. This loop receives commands from the user interface, and sends them directly to the message handler loop. The commands are sent over the network using Network Streams.
30 ni.com
Sending Commands Across the Network
• Network Streams are great for commands because are they are lossless
• Tips: • Use Flush with zero timeout to minimize latency • Add code to handle UI disconnections (see Sample Project)
Presenter
Presentation Notes
Network streams are similar to a Queue of an RT FIFO, except that they have network scope. They are a great option for sending commands or messages because they are lossless. Data is never overwritten or regenerated. Also, they are very reliable. Network stream functions will perform reconnections automatically in the background if the connection drops. They will retry forever. Network streams were designed for the streaming use case, so by default they prioritize throughput over latency. You can change this by calling the “Flush Stream” function with a timeout of zero, after writing an element to the string. Without this, the Network Stream function will sit there and wait for the buffer to fill up, in order to send a chunk of data at once (again, to maximize throughput for the streaming case). The flush stream function will ensure that your message is sent immediately after it is placed into the queue.
32 ni.com
System Monitoring
Disk Space RAM CPU Bandwidth
EFF
EC
T
Lost Data Crash Starvation
Embedded processors have limited….
Presenter
Presentation Notes
The last loop within the real-time application that we will talk about, is the System monitoring loop. So first, why is a system monitoring loop important? Embedded hardware targets are typically much more resource-constrained than with desktop PCs. Disk space, RAM, and CPU bandwidth are typically limited. You need to monitor these resources when developing and testing applications to ensure that you don’t run into limits that cause negative side effects.
33 ni.com
System Monitoring
Current value data can be sent to UI with Shared Variables
Monitor CPU usage per core Execute loop periodically
Presenter
Presentation Notes
Our system monitoring loop periodically checks CPU bandwidth and contiguous memory, and displays that data to the user interface. We can monitor CPU bandwidth by using the RT Get CPU Loads function from the LabVIEW Real-Time Utilities palette. We can also monitor usage per core, although in this example, our cRIO system only has one core. We generally recommend that you keep the CPU Bandwidth to below 80%. For a dual core system, we recommend that you keep at least one core below 80%. We are also monitoring the largest amount of contiguous memory using the System Configuration API. This API provides some of the functionality that you see when using Measurement and Automation Explorer to configure your hardware. Generally you want the contiguous memory, represented here in Bytes, to stay flat. If you see this number trailing downwards, then it is a cause for concern. At that point, you should review your code and find any instances where memory is being allocated dynamically. The LabVIEW for CompactRIO Developers guide provides some tips on how to overcome issues with dynamic memory allocations. Notice that the CPU Usage and Contiguous Memory data is being sent over the network using Network Published Shared Variables, versus Network Streams which we discussed earlier for sending commands. Network Published Shared Variables are a better fit for this type of data transfer since we are only concerned with the latest value, and we are sending the data periodically.
34 ni.com
Network Published Shared Variables
• When to use the Static API: • Small number of variables (less than a dozen)
• When to use the Programmatic API: • Iterate through a large number of variables • Dynamically change the IP address of the cRIO from the client
(client side only)
Presenter
Presentation Notes
When using Network Published Shared Variables, you have two APIs to choose from. Previously you saw the static API, this is where we drop down Shared Variable Nodes that we can instantly read and write to. This works best for a small number of variables – generally less than a dozen. When you have more than a dozen variables, we recommend using the programmatic API. The programmatic API allows you to iterate through a large number of variables, which makes for a cleaner, and more scalable block diagram. Another example of when you might consider using the programmatic API, is if you are in a situation where you need to dynamically change the IP address of a cRIO from the user interface. For example, in our CompactRIO Sample Projects, we provide a user interface VI that can connect to multiple cRIOs, one at time, depending on what IP address the user has specified on the front panel. In this example you will need to update the IP address of the Shared Variable Library you are connecting to, with the new IP address that the user just typed in. When using static shared variable nodes, the IP address is statically configured in the LabVIEW project, so this would not be possible.
35 ni.com
Basic Real-Time Processor Diagram
Periodically communicates tags
Communicates messages with client
Sends messages to other processes or
targets
Presenter
Presentation Notes
Review of the real-time processor diagram.
36 ni.com
Basic HMI Diagram
Generates UI Events using the Event Structure
Processes UI Events and communicates messages with real-time system
Periodically updates UI
Presenter
Presentation Notes
Review of the HMI.
37 ni.com
Common Variant Architectures
• Other LabVIEW for CompactRIO Sample Projects • LabVIEW FPGA Control • LabVIEW FPGA Control with Real-Time Sequencer Engine • LabVIEW Real-Time Control (RIO Scan Interface) • LabVIEW FPGA Waveform Acquisition and Logging • LabVIEW Data Logging and Supervisory Control
38 ni.com
FPGA Control with Sequencer Sample Project
39 ni.com
Real-Time Control Sample Project
Presenter
Presentation Notes
Take off FPGA VI
40 ni.com
FPGA Waveform Acquisition and Logging
Presenter
Presentation Notes
Are we streaming data to Client using NS?? Double check this
41 ni.com
Demo – LabVIEW FPGA Control Sample Project
42 ni.com
LabVIEW for CompactRIO Sample Projects
• Available in LabVIEW 2012 and later • Find more best practices at ni.com/compactriodevguide
