In this presentation, we propose an SPDK  user space vhost-user-fs solution, which can be used to accelerate file access in VMs and Containers.

We will present this solution in detail including the utilization of techniques such as virtio-fs, blobfs. Relying on this solution,

we are going to build a fast, consistent and secure manner to share directory tree on host to guests.

Live recovery is very useful in production environment, it can let users upgrade their vhost process without interrupting VMs, in this presentation,

We will introduce this new feature.

MemVerge is a startup based in Silicon Valley, focusing on building the next generation high performance data infrastructure based on persistent memory. In this presentation, we would start by briefly introducing some of the data infrastructure software that MemVerge has been working on followed by discussing a few use cases we are currently exploring. As an avid user of PMDK, we would like to further share a few lessons learned during our product development, where some product components are being actively prototyped with PMDK. In particular, we would like to talk about our experience using PMDK’s memory allocator. We discuss a few memory fragmentation challenges that we encountered, and present our solutions based on allocation class and the control functions on memory arena that was recently introduced in PMDK 1.6.

In the deep learning scenario, video and graphics are trained by GPU, which requires high concurrency and random reading of a large number of small data blocks. Under the common HDFS storage architecture, the performance of small files and the utilization of storage space are both low. High-speed GPU computing components need to match high-speed NVMe storage, HDFS storage architecture is difficult to unleash NVMe storage’s extreme performance.

Baidu Chitu Storage provides high throughput and low latency shared NVMe storage for GPU cluster, and adopts a layered architecture. The lower layer is a distributed high-availability block-level logical volume system based on NVMe-oF, and the upper layer is a parallel file system called Baidu Parallel FS (bpfs) on the logical volume. Small files and their meta-information are packaged and stored in logical volumes. The client accesses small files directly through NVMe-oF to the remote storage server. SPDK NVMe-oF is the key module of data path.

Baidu Chitu's storage latency is 10us higher than local NVMe. On one single client, multithreaded random IO of 16K small file can saturate the 100G network, and 500K IOPS for small files. Before reaching the hardware bottleneck, the aggregation throughput of multi-storage servers increases linearly. Meanwhile the aggregation throughput of multi-client random reads also increases linearly and the latency remains unchanged. Test performance from video and image trainings of 2 billion files is equivalent to the local multi-NVMe RAID0. Under the pressure of millions of IOPS on storage servers, using SPDK NVMe-oF saves 8 times CPU overhead than using Linux kernel modules. At the same time, the development and operation efficiency of SPDK user-mode program is much higher than that of kernel module.

Take a deep dive into the details of profiling to optimize performance with persistent memory.  If you want to take advantage of the Intel® Optane® DC persistent memory in AppDirect mode then PMDK library is probably your best bet. But what can you do if the performance you get doesn’t satisfy you? In this talk you will learn how to use Intel® VTune™ Amplifier to optimize PMDK-based applications.

The goal of integrating SPDK in our NAS system is to build a highly available and high performenc cache layer with low latencies. We will use SPDK to take over local nvme device and export it out with SDPK iscsi target or NVMe-OF of the SPDK for higher performence. Then in our gateway nodes ,one local nvme device and one remote nvme device can be made mirrors as a cache layer in our NAS system.In this talk, we will go over the cache layer design of our NAS system,and how we use the SPDK to use remote nvme device in our system. 

The capacity of data grows rapidly in big data area, more and more memory are consumed either in the computation or holding the intermediate data for analytic jobs. For those memory intensive workloads, end-point users have to scale out the computation cluster or extend memory with storage like HDD or SSD to meet the requirement of computing tasks. For scaling out the cluster, the extra cost from cluster management, operation and maintenance will increase the total cost if the extra CPU resources are not fully utilized. To address the shortcoming above, Intel Optane DC persistent memory (Optane DCPM) brings the break to the traditional memory/storage hierarchy and scale up the computing server. It brings higher capacity than memory and higher bandwidth & lower latency than storage like SSD or HDD. And Spark is widely used in the analytics like SQL and ML on the cloud environment. For cloud environment, low performance of remote data access is typical stop gap for users especially for some I/O intensive queries. For the ML workload, it’s an iterative model which I/O bandwidth is key to the end-2-end performance. In this talk, we will introduce how to accelerate Spark SQL with OAP (https://github.com/Intel-bigdata/OAP) to accelerate SQL performance on Cloud to archive 8X performance gain and RDD cache to improve K-means performance leveraging Intel Optane DCPMM with 2.5X performance improvement. Also we will have a deep dive about the root cause for those performance gains.

Intel  Optane DC persistent memory can be configured either as persistent memory (AppDirect) or as main memory, with DRAM used as a cache (memory mode).  Each mode has some challenges for adoption and to extract the best performance from this memory technology and identify which configuration mode is best suited for your application, it is necessary to understand the architectural flow from the core to the memory and some key glass jaws. In this presentation, we will present an overview of the uncore architecture leading to architectural glass jaw issues to monitor when using Intel Optane DC persistent memory and the conditions to monitor when using Intel Optane DC persistent memory in both memory configurations with the relevant architectural background, recommendation on what tools/profiles to use with customer applications.

Intel  Optane DC persistent memory can be configured either as persistent memory (AppDirect) or as main memory, with DRAM used as a cache (memory mode).  Each mode has some challenges for adoption and to extract the best performance from this memory technology and identify which configuration mode is best suited for your application, it is necessary to understand the architectural flow from the core to the memory and some key glass jaws. In this presentation, we will present an overview of the uncore architecture leading to architectural glass jaw issues to monitor when using Intel Optane DC persistent memory and the conditions to monitor when using Intel Optane DC persistent memory in both memory configurations with the relevant architectural background, recommendation on what tools/profiles to use with customer applications.

With traditional interrupt driven I/O, the CPU is either doing something useful or waiting.  With SPDK’s polled I/O the CPU is always 100% busy so traditional profiling techniques don’t work.  Intel® VTune™ Amplifier can identify “empty” spinning so you can balance core loading, balance SSDs, see the throughput per device, PCIe traffic breakdown and lots of good stuff.  Learn how to use Intel VTune Amplifier to optimize your I/O performance.

SSD is becoming ubiquitous in both Client and Data Center markets. The requirements on function, performance and reliability are refreshed frequently. As a result, SSD design, especially the firmware, has been keeping upgrading and restructuring for the decade. 

The test makes the change under control. However, the firmware test is not as mature as the software test. We have well developed methodologies, processes and tools for software. But the embedded platform, where the firmware executes, only provides the limited resources on computation and memory. So, it is difficult to run full test in the native embedded environment. Practically, SSD vendors run system tests with 3-rd party software, consuming huge resources. The existed tools lacks the flexibility to make efficient tests against vendor's own features and flaws. SSD developers need an infrastructure to implement their test s or programs in low cost. Our pynvme is just the answer. 

The test-dedicated light-weighted NVMe driver is the most essential part of the solution, where we rely on SPDK. First, SPDK is reliable. It is designed and tested in large-scale Data Center. Second, SPDK is highly modularized. We can choose modules we need, and extend them with our own features. Last, but not the least, SPDK is active and open. People work for better quality and latest features, so we can focus on the features for testing. I even forget to mention the best-in-class performance.

As part of the SPDK community, pynvme also contributes to the up-stream. And another interesting side-effect is that, since SSD devices pass the test of pynvme s, they also pass the very first test of SPDK! 

• Latest progress done with NVMe-OF RDMA performance.

• Introduce initial proposal to extend the internal POSIX-like transport API to allow better integration with zero-copy enabled TCP stacks, such as Mellanox’s VMA.

• Explain how Mellanox T10-DIF offload can be used to add integrity functionality to NVMe-OF protocol at initiator and target sides Including adding/striping/verifying T10-DIF or simple CRC.

With persistent memory, data can be retained after a program crash or power failure. In this session, learn how to make your C++ application persistent memory aware using the Persistent Memory Developers Kit (PMDK). The presentation includes C++ code samples walkthrough.