Distribution of WXXM data in the NAS using OGC data access services

Enabling Information Sharing
thru Common Services
Distribution of WXXM data in
the NAS using OGC data
access services
Presented To: Services Session
Presented By: Oliver Newell
Date:
August 31, 2011
This work was sponsored by the Federal Aviation Administration under Air
Force Contract No. FA8721-05-C-0002. Opinions, interpretations,
conclusions, and recommendations are those of the authors and are not
necessarily endorsed by the United States Government.
Introduction
• This talk covers
• High-level architecture concepts for distribution of
weather data in the NAS, based on R&D conducted
by the NNEW and SWIM programs
• Exercising the architecture concepts using an OGC
Web Feature Service (WFS) to disseminate WXXM
data
Agenda
• Architectural Background
• Distribution of WXXM data using the NNEW
Web Feature Service Reference
Implementation (WFSRI)
• Summary
JPDO Integrated Surveillance
Concept of Operations
Interagency shared situation awareness (SSA) enabled by network access
to shared services and collaborative capabilities built into C2 systems
Internet
Public Access Portal
•Filter
•Delay
DOT C2 Facility
DOT C2 Facility
(DOT-specific
fns.)
DOT C2 Facility
(DOT-specific
fns.)
(DOT-specific fns.)
DoD C2 Facility
DoD C2 Facility
(DoD-specific
fns.)
DoD C2 Facility
(DoD-specific
fns.)
(DoD-specific fns.)
DHS C2 Facility
(DHS-specific fns.)
NOAA and Other
NOAA
and Other
Facilities
Facilities
Net Centric Data
Distribution
Shared Surveillance Sources
Shared Services
•SSA Data Management
•SSA Correlator/Tracker
•Weather Data Reduction
•Other
Interoperability via WXXM
WXXM 1.1 Data Model (Collaborative effort among
Eurocontrol, FAA, NWS, DoD, NOAA)
Standards Governance Body
(* = Notional)
Agility
High
(months)
Eurocontrol-Specific
Extensions
FAA-Specific
Extensions
NWS-Specific
Extensions
DOD-Specific
Extensions
Aviation-Specific
Weather
Components (WXXM 1.1)
General-purpose
WXXM
Components
General-purpose
Weather Components (WXXM 1.1)
Observations
& Measurements
Individual
Organizations
International Civil
Aviation Organization (ICAO) *
World Meteorological
Organization (WMO) *
Observations & Measurements / ISO 19123
GML
ISO / OGC
ISO 19139
Low
(years)
XML
W3C
•• Composable,
Composable, extensible
extensible data
data model
model balances
balances standardization
standardization with
with the
the need
need for
for individual
individual
communities
communities to
to innovate
innovate over
over time
time
•• What
What is
is the
the vision
vision for
for distributing
distributing this
this data
data within
within the
the NAS?
NAS?
NextGen Standards and
Programs 'Stack'
NextGen Enterprise Architecture System View 4 (SV-4)
Interaction Services
Thick Clients
Web Clients
Mission Services
Short-term
Capacity
Management
Long-term
Capacity
Management
Navigation
Support
Trajectory
Management
Safety
Management
Support Services
Content Management
Weather (NNEW)
Data Access
Surveillance
Data Access
Content
Discovery
Weather
Data Access
Aeronautical
Data Access
FTI
SOA Core Services
Messaging
Interface
Management
Service
Discovery
Publish/
Subscribe
Request/
Response
Message
Routing
Technical Infrastructure Services
Hardware
Computing
Platforms
SWIM
Hosting
Platforms
Network
Data
Storage
Terrestrial
Comm
Air/Ground
Comm
Weather Data Dissemination Supporting
Infrastructure Programs
Key Challenge – Efficient use
of Network Bandwidth
FTI Optical Backbone Segment
ZSE
Olympia
ZMP
VNT
ZBW
Albany LVLT
r
Chepachet
Stamford
AC
Y
DCC
ZOBChester
Reno
ZA
U
a
Oakland
ZLC
ZOA
ZNY
Pittsburgh
a
ZDV
ZID
ZD
C
Frederiksberg
Cincinnati
Richmond
Chesapeake
ZKC
o
Raleigh
G
Charlotte
Charlotte
OEX
ZL
A
ZME
.
ZAB
Little Rock
Nashville
ZTL Green Ville
Columbia
Segment
••
••
••
ZF
W
Augusta
Savanna
ZJX
FTI
POP
Weather
be
compared
surveillance
Weather data
data can
can
be large
large in
in size
size when
when
compared with
with typical
typicalOrlando
surveillance data
data
2010 Core
ZHU
1000's
many
1000's
of users
users ––Rings
many getting
getting the
the same
same or
or similar
similar data
data
OC-N
‘s of
Expansion
FTI
not
(currently)
encouraging
use
of
multicast
protocols
layer
ZMA
FTI not (currently)
encouraging
use of multicast protocols at
at network
network
layer due
due to
to
Rings
(20102015)
management
management complexity
complexity
Tampa
Melbourne
It's Not a Brand-New Problem...
Scalable Web Caching Architecture
•• To
To this
this concept,
concept, we
we would
would like
like to
to add:
add:
----
Spatial
Spatial filtering
filtering operations
operations relevant
relevant to
to aviation
aviation weather
weather
Support
Support for
for publish/subscribe
publish/subscribe message
message exchanges
exchanges
Common
Common interface
interface semantics
semantics for
for real-time
real-time and
and archived
archived data
data
OGC Data Access Services
International standards for access to data of all types using spatial/temporal queries
• Web Coverage Service – Gridded data access
• Web Feature Service – Non-gridded data access
2 spatial dimensions + time
3 spatial dimensions + time (4-D Cube)
•• OGC
OGC data
data access
access services
services not
not historically
historically focused
focused on
on continuous
continuous real-time
real-time data
data delivery
delivery
•• NNEW
NNEW is
is extending
extending WCS/WFS
WCS/WFS to
to include
include publish/subscribe
publish/subscribe capability
capability
-- Goal
is
to
standardize
the
extensions
within
OGC
(Pub/Sub
Goal is to standardize the extensions within OGC (Pub/Sub Working
Working Group)
Group)
Service-Oriented Weather
Content Delivery Network
NAS
NAS
Enterprise
Enterprise
Gateway
Gateway
Internal NAS
Weather Data
Origin Server
FAA
•• Content
Content Delivery
Delivery Network
Network (CDN)
(CDN) for
for
Weather
Data
Weather Data
−− Origin
Origin Servers
Servers
−− Distribution
Distribution Servers
Servers
−− Common
Common Interfaces
Interfaces at
at all
all Server
Server Tiers
Tiers
(OGC
WFS,
WCS,
WMS)
(OGC WFS, WCS, WMS)
−− Leverages
Leverages and
and extends
extends SWIMSWIMcompliant
compliant Pub/Sub
Pub/Sub Messaging
Messaging
External
Weather Data
Origin Servers
(NWS, …)
• Top-Tier distribution servers isolate
origin servers from demands of
multiple distributed clients
Top-Tier
Distribution Servers
Large Site:
Many Weather
Data Clients
Edge
Distribution
Server
Small Site:
Few
Weather
Data
Clients
• Edge distribution servers isolate
network and top-tier servers from
demands of many clients at a single
location (e.g., large TRACON facility)
• Actual deployed distribution server
topology can grow or shrink over time
based on changing user demand and
underlying network capability
Agenda
• Architectural Background
• Distribution of WXXM data using the NNEW
Web Feature Service Reference
Implementation (WFSRI)
• Summary
Roles of the NNEW Service
Reference Implementations
• Proof-of-concept platform used to exercise
the core OGC specifications and extensions
against NNEW requirements
• Build on top of the SWIM software stack provide lesson’s learned
• Technology transfer from R&D to NNEW
implementation phase (Government
Furnished Information)
WFSRI Block Diagram
Service
Service Container
Container (e.g.,
(e.g., Tomcat,
Tomcat, Jetty,
Jetty, ServiceMix)
ServiceMix)
SQL Database
Apache Derby, Postgres,…
Data
Store
Triggered
Subscription
Query
Requests
Insert
Single-Shot
Query
Responses
Single-Shot
Query
Requests
Single-Shot
Requests/Responses
Feature
Cache
TransactionInsert
(SOAP/HTTP)
Request/
Response
Transaction
Interface
(Insert)
OGC
Query
Processor
Spatial Filter
Java Topology
Suite
XML
Feature
Parsing
Jetty/
Apache
Camel
New
Data
Trigger
Request/
Response
Query
Interface
Jetty/
Apache
Camel
Subscription
Manager
Subscription
Subscription
Subscription Setup
Requests/Responses
Monitoring ‘Beans’
Remote Dataset
Subscription Data
(JMS)
Pub/Sub
TransactionInsert
(JMS)
Repeater
Insert
Insert
Subscription
Data
Message Broker
Apache ActiveMQ
JMX
Mbean
Server
Monitoring
Monitoring
‘Beans’
‘Beans’
Subscription
Data
WFSRI Installation Verification/
Performance Measurement
Sample
Sample program
program capable
capable of
of
simulating
a
variety
simulating a variety of
of products
products
at
at aa number
number of
of different
different spatial
spatial
densities
densities
Web
WebFeature
FeatureService
Service
Reference
ReferenceImplementation
Implementation
SamplePublisher
SamplePublisher
Simulator
Test
Files
Sample
Core
TransactionInsert
(SOAP)
SampleReqReply
SampleReqReply
GetFeature
(SOAP)
Sample
Core
WFSRI
Core
Subscription
Control/Setup
(SOAP)
(CmdLine
Interface
&
I/O)
SampleSubscribe
SampleSubscribe
rr
TransactionInsert
(JMS)
•• Data
Data flow
flow instrumented
instrumented to
to enabled
enabled
creation
of
timing
trace
at
creation of timing trace at end
end receiver
receiver
•• HTTP,
HTTP, JMS
JMS headers
headers used
used (similar
(similar to
to
HTTP
HTTP 'Via'
'Via' header
header in
in principle)
principle)
Message
Broker
Sample
Core
Subscribe
(JMS)
Hostname
AppName
After
Hostname
AppName
After Timestamp(ms)
Timestamp(ms) DeltaTime
DeltaTime
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------sampson.local
WFS
Recv
sampson.local
WFS
Recv 1314164845047
1314164845047
localhost
WFS
Proc
localhost
WFS
Proc 1314164845455
1314164845455 (( 408
408 ms
ms ))
localhost
WFS
Zip
1314164845469
(
14
ms
localhost
WFS
Zip
1314164845469 ( 14 ms ))
sampson.local
WFSClientApp
sampson.local
WFSClientApp Recv
Recv 1314164845471
1314164845471 (( 22 ms
ms ))
sampson.local
WFSClientApp
sampson.local
WFSClientApp Unzip
Unzip 1314164845481
1314164845481 (( 10
10 ms
ms ))
Simulated Storm Contours
(Dense Scenario – NW Filter)
WFSRI Installation Verification/
Performance Measurement
•• ‘Dense’
‘Dense’ simulation
simulation exceeds
exceeds feature
feature count
count of
of CIWS
CIWS worst-case
worst-case weather
weather
day
day
•• ~~ 22 seconds
seconds per
per distribution
distribution node
node ‘hop’
‘hop’ for
for this
this loading
loading is
is not
not considered
considered
problematic
problematic (forecast
(forecast data
data latency
latency requirements
requirements are
are relaxed
relaxed when
when
compared
compared to
to their
their wind-shear
wind-shear alert
alert counterparts)
counterparts)
•• Room
Room for
for improvement
improvement remains
remains via
via aa number
number of
of optimizations
optimizations
WFS, WXXM, and AIXM
Querying for weather in an Airspace
Volume
•WFS issue: no generic GML ‘2.5-D’ data types
(2-D shapes plus vertical extent)
•AIXM provides the needed types
<AirspaceVolume>
<AirspaceVolume>
<upperLimit
<upperLimit uom="m">5000</upperLimit>
uom="m">5000</upperLimit>
<upperLimitReference>STD</upperLimitReference>
<upperLimitReference>STD</upperLimitReference>
<lowerLimit
<lowerLimit uom="m">0</lowerLimit>
uom="m">0</lowerLimit>
<lowerLimitReference>MSL</lowerLimitReference
<lowerLimitReference>MSL</lowerLimitReference
<horizontalProjection>
<horizontalProjection>
<Surface
<Surface gml:id="SURF">
gml:id="SURF">
<gml:patches>
<gml:patches>
<gml:PolygonPatch>
<gml:PolygonPatch>
<gml:exterior>
<gml:exterior>
<gml:LinearRing>
<gml:LinearRing>
<gml:posList
<gml:posList dimension="2">
dimension="2">
40.450001
40.450001 -93.900002
-93.900002
40.525002
40.525002 -93.491669
-93.491669
(additional
(additional points....)
points....)
40.450001
40.450001 -93.900002
-93.900002
</gml:posList>
</gml:posList>
</gml:LinearRing>
</gml:LinearRing>
</gml:exterior>
</gml:exterior>
</gml:PolygonPatch>
</gml:PolygonPatch>
</gml:patches>
</gml:patches>
</Surface>
</Surface>
</horizontalProjection>
</horizontalProjection>
</AirspaceVolume>
</AirspaceVolume>
WFS AIXM Support Example
<wfs:WFS_Capabilities>
<wfs:WFS_Capabilities>
…
…
<fes:Spatial_Capabilities>
<fes:Spatial_Capabilities>
<fes:GeometryOperands>
<fes:GeometryOperands>
<fes:GeometryOperand
<fes:GeometryOperand name="gml:Point"/>
name="gml:Point"/>
<fes:GeometryOperand
<fes:GeometryOperand name="gml:Curve"/>
name="gml:Curve"/>
<fes:GeometryOperand
<fes:GeometryOperand name="gml:Polygon"/>
name="gml:Polygon"/>
<fes:GeometryOperand
<fes:GeometryOperand name="gml:Envelope"/>
name="gml:Envelope"/>
<fes:GeometryOperand
<fes:GeometryOperand name=“aixm:AirspaceVolume"/>
name=“aixm:AirspaceVolume"/>
</fes:GeometryOperands>
</fes:GeometryOperands>
<fes:SpatialOperators>
<fes:SpatialOperators>
…
…
</wfs:WFS_Capabilities>
</wfs:WFS_Capabilities>
<wfs:GetFeature>
<wfs:GetFeature>
<wfs:Query
<wfs:QuerytypeNames="avwx:PIREP">
typeNames="avwx:PIREP">
<fes:Filter>
<fes:Filter>
<fes:Within>
<fes:Within>
<fes:ValueReference>GEOMETRY</fes:ValueReference>
<fes:ValueReference>GEOMETRY</fes:ValueReference>
<aixm:AirspaceVolume>
<aixm:AirspaceVolume>
<aixm:upperLimit
<aixm:upperLimit uom="m">5000</aixm:upperLimit>
uom="m">5000</aixm:upperLimit>
<aixm:upperLimitReference>STD</aixm:upperLimitReference>
<aixm:upperLimitReference>STD</aixm:upperLimitReference>
<aixm:lowerLimit
<aixm:lowerLimit uom="m">0</aixm:lowerLimit>
uom="m">0</aixm:lowerLimit>
<aixm:lowerLimitReference>MSL</aixm:lowerLimitReference>
<aixm:lowerLimitReference>MSL</aixm:lowerLimitReference>
<horizontalProjection>
<horizontalProjection>
<Surface
<Surface gml:id="SURF">
gml:id="SURF"> ….
…. (details
(details omitted)</Surface>
omitted)</Surface>
<aixm:AirspaceVolume>
<aixm:AirspaceVolume>
</fes:Within>
</fes:Within>
</fes:Filter>
</fes:Filter>
</wfs:Query>
</wfs:Query>
</wfs:GetFeature>
</wfs:GetFeature>
WFS Weather Contour Query
Filtered by AIXM <AirspaceVolume>
FAA Tech Center R&D Enclave
(Weather Domain Portion)
•Data
•Data providers
providers pass
pass variety
variety of
of weather
weather
data
products
via
R&D
NAS
Enterprise
data products via R&D NAS Enterprise
Gateway
Gateway (NESG)
(NESG)
•Flexible
•Flexible environment
environment -- ability
ability to
to
simulate
simulate multiple
multiple data
data distribution
distribution tiers
tiers
•Quality-of-Service
•Quality-of-Service (QoS)
(QoS) capable
capable
routers
between
tiers
routers between tiers
•WFS/WCS
•WFS/WCS reference
reference implementations
implementations
currently
being
deployed
currently being deployed and
and
instrumented
instrumented
•Focus
•Focus on
on performance,
performance,
documentation
documentation of
of lessons
lessons learned
learned
Summary
• FAA has a need to efficiently distribute WXXM
data in the operational NAS environment
• The OGC WFS interface, augmented with
pub/sub extensions and configured in a hub &
spoke topology, can be used to help
implement this vision
• NNEW WFS Reference Implementation being
used to demonstrate feasibility of approach
Questions & Answers /
Feedback
More Information
/ Contacts
• WFS/WCS Reference Implementations
-
https://wiki.ucar.edu/display/NNEWD/Reference+Implementations
• olivern@LL.mit.edu