IRNC AMIS Project

Overview

AMIS: Advanced Measurement Instrument and Services aim to achieve flow-granularity network measurement, sustain scalable line rate, meet evolving measurement objectives and derive knowledge for network advancement. The scalable hardware architecture and software-defined measurement framework are key advantages of AMIS. The thrust of the work is to prototype, deploy and apply an advanced measurement instrument and to enable services for accurate network monitoring and in-depth traffic analysis at exchange points of international R&E networks. The instrument will support flow-granularity measurement and monitoring at line rates at up to 40Gbps, and engage software APIs to examine selected flows, with little impact to the performance of user traffic.

Besides direct support of existing measurement tools (e.g., perfSONAR), the proposed instrument will provide libraries and applications allowing project-level or Autonomous System (AS)-level flow statistical queries. With scalable hardware and an open source software stack, the measurement services will equip network operators with effective tools to quantify flow-level network performance, and more importantly to enable in-depth flow analysis through software libraries. Such services bring opportunities for big data analysis of measurement results and packet traces. The knowledge derived from the measurement will benefit network operation and provisioning.

Objectives

The major objectives of AMIS project include the following:

Design a hardware platform with scalable processing capabilities on network flows at 10, 40, and 100Gbps line rate;
Design a software defined measurement framework that allows network operators to specify and instantiate measurement tasks and query results;
Design privacy preserving algorithms to report measurement results while protecting the user privacy;
Manage and analyze measurement data, make observations and provide insights about network usage patterns for network planning and operation.

Current Deployment

As of May 2019, we have deployed the distributed measurement prototypes at five IRNC locations: UMass Lowell, Univ. of Kentucky, ICAIR of Northwestern University, Florida International University/AMPATH, La Serena, Chile. The AMIS instrument at these five locations are fed with 40Gbps or 100Gbps synthetic and real-world network traffic and are operating to serve as testbeds for the designed hardware architecture and software framework.

Project Team & Partners

Project Team:

Yan Luo, PI, University of Massachusetts Lowell
Cody Bumgardner, Co-PI, University of Kentucky
Gabriel Ghinita, Co-PI, University of Massachusetts Boston
Michael McGarry, Co-PI, University of Texas El Paso

Other partners:

Northwestern University
Florida International University
Intel Corporation
Indiana University

AMIS Framework

Measurement Functions

The measurement substrate provides advanced capabilities to measure fine-grained network flows at up to 100Gbps line rate. The measurement functions include for examples, packet loss detection, packet count, top k talker, and Netflow generation. The functions are specified with EQuery language and composed to programmable hardware.

Contributed by team at UMass Lowell. Learn more about measurement functions

Privacy Preserving

Protecting data privacy is challenging, there is no “one-fits-all” privacy approach. Choosing approach depends on intended data use, accuracy constraints, performance constraints etc. The AMIS Privacy Framework is a comprehensive framework that encapsulates most prominent privacy models including syntactic models (generalization/suppression) and semantic models (differential privacy) and so on, supporting a diverse set of data uses (e.g. operations, statistics, data mining)

Contributed by team at UMass Boston. Learn more about privacy preserving

Data Visualization and Analytics

Visualization and analytics of measurement data are important to operators. The AMIS framework annotates instrument data with auxiliary data for analytics and provides data visualization and analytics to support network management.

Contributed by team at Univ. of Texas at El Paso. Learn more about data analytics

Plugin Management

The components of AMIS are managed as plugins. The plugin management help config management, dispactch measurement tasks to AMIS instruments, query processing on netflow records and so on.

Contributed by team at Univ. of Kentucky. Learn more about Privacy Preserving

Demo

We have demonstrated the prototypes of the AMIS framework at past International Conference for High Performance Computing, Networking, Storage and Analysis (Supercomputing) 2017 and 2018 together with our collaborators. The most recent demo of the project was at IFIP/IEEE IM 2019: IFIP/IEEE International Symposium on Integrated Network Management 2019, April 8-12, Washington DC, where the team won the Best Experience Paper Award.

IRNC AMIS project is sponsored by the National Science Foundation of USA under Award No. 1450996, 1450937, 1450975 and 1450997.