This repository provides OpenAPI 3.0 specifications and raw data schemas for the Network Intelligence Framework (NIF) subsystems, enabling developers to integrate and implement network intelligence capabilities in their own services.
The Network Intelligence Framework (NIF) provides application programming interfaces (APIs) for both network AI model developers and on-demand monitoring users. These APIs support a variety of operations including on-demand data collection, preprocessing, data management and sharing, insight extraction, and AI platform orchestration. NIF is designed to enable the seamless flow of network data through a pipeline from raw packet capture to AI-driven insights.
The architecture of NIF comprises four core subsystems:
ODMSS serves as the front-end interface for receiving data collection requests from internal or external systems. It orchestrates the Agent Subsystem to collect flow-level or packet-level raw data, stores it in the Repository and Preprocessing Subsystem, and facilitates ingestion from legacy systems via standardized interfaces.
AGTSS is responsible for intercepting packet information from the network to generate raw datasets based on on-demand specifications. It operates under the control of ODMSS, forwarding the collected data accordingly. AGTSS supports monitoring of traditional physical network links as well as virtualized network environments (e.g., traffic from VMs or containers in NFV architectures).
This subsystem handles preprocessing, labeling, and storage of data. It supports the automation of preprocessing pipelines and maintains a scalable big-data infrastructure capable of parallel processing. The repository stores raw data, AI-ready datasets, and both pre-trained and fine-tuned network AI models. These models can be discovered and recommended based on user queries.
AIPSS provides a container-based machine learning environment tailored for deep learning with GPU-accelerated servers. It supports automated composition of model design and training pipelines, including hyperparameter optimization. AIPSS enables full-lifecycle automation from data collection to model training and retraining, and optionally provides an inference runtime environment.
This repository makes available OpenAPI 3.0 specifications for two key NIF subsystems:
OpenAPI specification for the On-Demand Monitoring Subsystem (ODMSS), enabling on-demand data collection operations. This API provides comprehensive endpoints for:
- Creating and managing data collection requests
- Querying collection status and results
- Handling collection events and callbacks
- Managing collection resources and agents
OpenAPI specification for the Repository and Preprocessing Subsystem (REPSS), enabling on-demand data preprocessing operations. This API provides endpoints for:
- Submitting preprocessing jobs
- Managing preprocessing orders and workflows
- Retrieving preprocessing results
- Handling preprocessing status notifications
Under ODMSS control, AGTSS produces three record families: flow data, flow unit-time data, and packet data. Users select which family (or families) to collect per order and may further restrict the fields within each family. The three data families offer different levels of granularity to serve various network analysis and AI training needs.
- Flow data (
raw_data_schema/raw_data_flow.avsc.json): Provides comprehensive per-flow statistics with 77 fields, ideal for flow identification, performance analysis, and TCP signaling analysis. - Flow unit-time data (
raw_data_schema/raw_data_flow_delta.avsc.json): Offers time-sliced aggregation with 59 fields, enabling trend analysis and real-time monitoring dashboards. - Packet data (
raw_data_schema/raw_data_flow_packet.avsc.json): Captures detailed packet-level information with 41 fields, supporting packet-level forensics and protocol behavior analysis.
Choose the appropriate family based on requirements for data granularity, storage efficiency, and analytical depth.
Sample datasets collected and processed using the NIF system from Busan Metropolitan City's administrative network traffic are publicly available. Five types of datasets can be downloaded from the Busan Big-Data Wave site, including circuit traffic data, peak traffic data, ultra-short cycle traffic data, and sampled flow/packet datasets. For detailed information about each dataset, including download links, field descriptions, and use cases, see OpenSampleDataset.md.
These OpenAPI specifications can be used as reference documentation for developing services similar to NIF ODMSS and REPSS. The specifications provide detailed information about:
- API endpoints and operations
- Request/response schemas
- Authentication and authorization mechanisms
- Callback and event notification patterns
- Error handling and status codes
Since these specifications follow the OpenAPI 3.0 standard, you can leverage automated code generation tools to create:
- Client SDKs: Generate client libraries in various programming languages (Python, JavaScript, Java, Go, etc.)
- Server Stubs: Generate server skeleton code for implementing the APIs
- API Documentation: Generate interactive API documentation
- Mock Servers: Create mock API servers for testing and development
Popular tools for OpenAPI code generation include:
- OpenAPI Generator
- Swagger Codegen
- swagger-codegen-cli
- Redoc for documentation
# Using OpenAPI Generator
npx @openapitools/openapi-generator-cli generate \
-i NIF_OpenAPIs/nif_odmss_openapi.yaml \
-g python \
-o ./generated/python-odmss-client
# Using Swagger Codegen
swagger-codegen generate \
-i NIF_OpenAPIs/nif_repss_openapi.yaml \
-l python \
-o ./generated/python-repss-clientNIF_OpenAPIs/
├── nif_odmss_openapi.yaml # ODMSS OpenAPI 3.0 specification
└── nif_repss_openapi.yaml # REPSS OpenAPI 3.0 specification
raw_data_schema/
├── raw_data_flow.avsc.json # Flow data schema (per-flow statistics)
├── raw_data_flow_delta.avsc.json # Flow unit-time data schema (time-sliced)
└── raw_data_flow_packet.avsc.json # Packet data schema (per-packet details)
This project uses a dual license arrangement: OpenAPI specifications, raw data schemas, documentation, and example code are available under Apache 2.0, while complete source code, implementations, and operational components require a Commercial License through technology transfer with ETRI. For details, see LICENSE.md.
For questions about NIF technology transfer, please contact ETRI.
- Chunglae Cho, clcho@etri.re.kr
- Jooyoung Lee, joolee@etri.re.kr
Note: This repository contains only the OpenAPI specifications and raw data schemas. For implementing similar services, refer to these specifications and use automated code generation tools. For the actual ODMSS and REPSS implementation code, please contact ETRI through a paid technology transfer arrangement.