The introduction of network-enabled applications to many defense and aerospace projects allows multiple participants to work cooperatively with shared data. Using file servers and Network Attached Storage (NAS), the applications can access and update the same data sets wherever they might be located on the network. Ethernet is the first choice of network technology for military and commercial/office applications of this type. However, there is a class of applications where Ethernet or other network/fabric technologies are just not fast or deterministic enough. Flight simulators, aircraft simulators, jet engine testing simulators, or war gaming/battlefield simulators are typical examples of complex systems requiring significant amounts of distributed computing power, sometimes spread over quite large distances and sharing large arrays of volatile data in real time. Many of these applications have turned to reflective memory because of its low latency and high degree of determinism.

Reflective memory principles

The basis of reflective memory operation is its ability to autonomously replicate the contents of one processor's memory to the memory nodes of all other network members. As a processor writes to memory, the location and content are propagated around a network to the same addressable location in all other network memory nodes. It is not necessary for the processor that is writing data into memory to use any protocol for the replication to take place; however, at the system level there is a need for some predetermination of ownership and access rights to locations within memory space. But this allocation is entirely user- or application-dependent. Higher layers of protocol such as semaphores or handshaking can be added, but again, only if required by a particular application's needs.

A typical application for reflective memory might be an aircraft simulator with processing nodes representing sensors, communications subsystems, flight systems, cockpit display systems, and so on. A number of these nodes would be writing data onto the network, which then replicates the data to all the other nodes. For example, a sensor node, such as a weather radar, will generate a stream of data representative of that particular sensor type, varying in real time with the attitude of the aircraft as it flies through the simulation scenario. With many sensor types and many subsystem types feeding data into such a distributed processing application, it is vital that latency and system response mirror the real-world environment for the simulation to be truly representative.

Low latency

Keys to the success of reflective memory are its low latency and determinism. Using reflective memory, the typical node-to-node propagation time is on the order of 1 microsecond. Thus, in a system of 30 nodes, it would take only 30 microseconds to propagate through the network to all nodes. Data is transferred at rates up to the available network bandwidth, which can be 170 MBps or more. Achieving the same latency using other network technologies, such as Ethernet, is impossible even using datagram broadcast because of IP protocol overheads, addressing, and memory write times.

Ring topology

Reflective memory networks use a high-bandwidth, ring topology with commonly used 2.12 Gbaud signaling to move data in dynamically variable packets of 4 to 64 bytes. Such a ring will have its own data transmission protocols and error checking/recovery mechanisms plus optional managed hubs, if required, to bypass faulty nodes. The network can be easily expanded simply by adding more memory nodes as required. Using fiber as the network media, node-to-node transmission distances up to 300 m using multimode fiber are possible, or up to 10 Km using single-mode fiber. Reflective memory appears as just a block of memory addressed by its host processor via PCI or PCI Express, enabling it to be used in heterogeneous processing systems with many different host and operating system combinations that support PCI or PCI Express. An example of such a reflective memory node with up to 256 MB of SDRAM, suitable for attachment to a host processor via PCI Express, is the PCIE-5565RC from GE Fanuc Intelligent Platforms (Figure 1).

Both large-scale aerospace/industrial simulators and smaller embedded simulators found within battle groups across a number of vehicles can benefit from the speed of response provided by reflective memory. Currently, no other technology offers the same levels of latency and determinism for use in high-performance, widely distributed, shared data processing applications.

To learn more, e-mail Duncan Young at young.duncan1@btinternet.com.