Conversation with the Testing Community

-

-

OPNFV testing community

OPNFV Design Summit, 20/6/2016, Berlin

-
- -

- -

- >“You make experiments and I make theories. Do you know the difference? A theory is something nobody believes, except the person who made it. An experiment is something everybody believes, except the person who made it." A.Einstein -

- -

- # Agenda - * Who are we? - * Upstream, Upstream, Upstream - * Towards Telco Cloud KPI -

- -

- # Who are we? - ![team](../../../img/OPNFV_testing_group.png) - This is a representation...not a governance! -

The test projects

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Test case	Description
Functest	Umbrella project for Functional testing
Yardstick	Umbrella project for performance/qualification testing
CPerf	SDN controller performance testing
StorPerf	Storage performance testing
VSPerf	VSwitch qualification
Bottlenecks	Detect possible bottlenecks
QTIP	Benchmark to boost performance

- -

- # Functest - * Functional testing to validate scenarios - * Contributors (raise your hands) - * Functest presentation (here...1h ago :)) - * breakout sessions - * Today 3.10PM-3.40PM (vIMS evolution) 4PM (feature project integration), 5PM (Colorado status) - * tomorrow 11AM (API, test collection), 1PM (work Upstream), 3.15PM (D Release)(to be confirmed...) -

- # Yardstick - * Performance testing framework and NFVI Validation - * [Contributors](https://wiki.opnfv.org/display/yardstick/People) - * Yardstick presentation - * How to Do a Pre-deployment NFVI Validation Quickly and Efficiently? Wednesday, June 22, 4:00 PM - 4:30 PM - * Yardstick: A Pre-deployment NFVI Validation Tool. Thursday, June 23, 3:10 PM - 3:30 PM - * breakout sessions - * Tomorrow 2PM (Yardstick in C) -

- # CPerf - * SDN Controller Performance Testing - * Upstream test experts + standards/metrics experts - * Goal: Cross-controller perf tests in CI against realistic deployments -

- # StorPerf - * Cinder Volume Performance Testing - * Based on SNIA's SSD Test Specification - * Presentation - * StorPerf: Cinder Storage Performance Measurement. - Wednesday, June 22, 3:40 PM - 3:55 PM - * Breakout Session - * Tomorrow 10:15 (StorPerf Test Results Working Session) -

- # VSPerf -

- # Bottlenecks -

- # QTIP - - ## OPNFV Platform Benchmark - - - Bottom up - - testing baremetal components first - - More than result - - comparing against reference - - Setup-Test-Diagnose-Improve - - a closed loop to boost platform performance - - ## Breakout session - - - 14:30~15:45, June 21st/Tuesday - - Room Tegel -

- # Upstream, upstream, upstream.. -

- ## We are on the shoulders of the giants - * Rally (OpenStack) - * ODL, ONOS, ... - * RobotFramework, Teston - * ..... -

- ## Time to give back - ### How to improve work with testing upstream community -

- -

- # Towards Telco Cloud KPI? -

- -

- # Testing group goals - * Define common needs (naming, rules, tools...) - * Unify result collection, reporting, dashboarding, documentation - * Share best practice: docker, CLI, .. - * Identify testing domains - * Encourage testing in poorly covered areas -

- -

- # The ultimate goal: define Telco Cloud KPIs - * Provide methodology and test suites to qualify a Telco Cloud - * Provide reference benchs for plugfest/third party integration: compare apple with apple -

- -

- # Thank you -

- -