Conversation with the Testing Community

OPNFV testing community

OPNFV Design Summit, 20/6/2016, Berlin

>“You make experiments and I make theories. Do you know the difference? A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it." A.Einstein

# Agenda * Who are we? * Upstream, Upstream, Upstream * Towards Telco Cloud KPI

# Who are we? ![team](../../../img/OPNFV_testing_group.png) This is a representation...not a governance!

The test projects

Test case	Description
Functest	Umbrella project for Functional testing
Yardstick	Umbrella project for performance/qualification testing
CPerf	SDN controller performance testing
StorPerf	Storage performance testing
VSPerf	VSwitch qualification
Bottlenecks	Detect possible bottlenecks
QTIP	Benchmark to boost performance

# Functest * Functional testing to validate scenarios * Contributors (raise your hands) * Functest presentation (here...1h ago :)) * breakout sessions * Today 3.10PM-3.40PM (vIMS evolution) 4PM (feature project integration), 5PM (Colorado status) * tomorrow 11AM (API, test collection), 1PM (work Upstream), 3.15PM (D Release)(to be confirmed...)

# Yardstick * Performance testing framework and NFVI Validation * [Contributors](https://wiki.opnfv.org/display/yardstick/People) * Yardstick presentation * How to Do a Pre-deployment NFVI Validation Quickly and Efficiently? Wednesday, June 22, 4:00 PM - 4:30 PM * Yardstick: A Pre-deployment NFVI Validation Tool. Thursday, June 23, 3:10 PM - 3:30 PM * breakout sessions * Tomorrow 2PM (Yardstick in C)

# CPerf * SDN Controller Performance Testing * Upstream test experts + standards/metrics experts * Goal: Cross-controller perf tests in CI against realistic deployments

# StorPerf * Cinder Volume Performance Testing * Based on SNIA's SSD Test Specification * Presentation * StorPerf: Cinder Storage Performance Measurement. Wednesday, June 22, 3:40 PM - 3:55 PM * Breakout Session * Tomorrow 10:15 (StorPerf Test Results Working Session)

# VSPerf

# Bottlenecks

# QTIP ## OPNFV Platform Benchmark - Bottom up - testing baremetal components first - More than result - comparing against reference - Setup-Test-Diagnose-Improve - a closed loop to boost platform performance ## Breakout session - 14:30~15:45, June 21st/Tuesday - Room Tegel

# Upstream, upstream, upstream..

## We are on the shoulders of the giants * Rally (OpenStack) * ODL, ONOS, ... * RobotFramework, Teston * .....

## Time to give back ### How to improve work with testing upstream community

# Towards Telco Cloud KPI?

# Testing group goals * Define common needs (naming, rules, tools...) * Unify result collection, reporting, dashboarding, documentation * Share best practice: docker, CLI, .. * Identify testing domains * Encourage testing in poorly covered areas

# The ultimate goal: define Telco Cloud KPIs * Provide methodology and test suites to qualify a Telco Cloud * Provide reference benchs for plugfest/third party integration: compare apple with apple

# Thank you