Going Stir Crazy? Expand Your PM Resume at These Virtual Events!

Jim Fister Leave a comment

We here at SNIA know that everyone is getting a tad stir crazy sitting at home. However, there are still some great opportunities to learn while you’re trying to decide which wall of the home office to face tomorrow. SNIA Compute, Memory, and Storage Initative (CMSI) member company Intel is offering some excellent resources for those interested in programming persistent memory using the open-source Persistent Memory Development Kit (PMDK).

Intel is hosting a virtual forum on PMDK, along with the Storage Performance Development Kit (SPDK), and vTune Profiler tools. This is a great opportunity to meet virtually with the teams who are developing the tools as well as the community building applications. The Virtual Forum runs June 23-35, with special focus on PMDK on June 25th. There are a variety of exciting sessions all three days.

Intel is also hosting two BrightTALK seminars on Persistent Memory. The first, Building Durable Storage Solutions with Intel Optane Persistent Memory on June 23rd, will focus on remote applications for persistent memory. Especially for those interested in networked storage solutions, this will be a great educational webinar. The second, Enabling Persistent Memory Usages in Cloud on June 30th, will cover how many of the most popular in-memory databases already take advantage of Persistent Memory.

In addition, SNIA is continuing to advance the Persistent Memory development conversation. We announced at the Persistent Memory Summit in January that SNIA would be exploring more opportunity for online development using Persistent Memory, as well as an Optane Memory Programming Challenge. Both of these will be active for the second half of this year, and you can watch this space for a formal announcement in the next month.  Learn about our successful NVDIMM Programming Challenge journey here.

Please feel free to register for the above events to learn more and join the community.

And may we suggest the north office wall for tomorrow?

Note: This has also been cross-posted at the PIRL Blog, a collaborative effort of the USCD Non-Volatile Systems Lab and SNIA. Go check out PIRL for some more Persistent Memory Development content.

Everyone Wants Their Java to Persist

Jim Fister Leave a comment

In this time of lockdown, I’m sure we’re all getting a little off kilter. I mean, it’s one thing to get caught up listening to tunes in your office to avoid going out and alerting your family of the fact that you haven’t changed your shirt in two days. It’s another thing to not know where a clean coffee cup is in the house so you can fill it and face the day starting sometime between 5AM and Noon. Okay, maybe we’re just talking about me, sorry. But you get the point.

Wouldn’t it be great if we had some caffeinated source that was good forever? I mean… persistence of Java? At this point, it’s not just me.

Okay, that’s not what this webinar will be talking about, but it’s close. SNIA member Intel is offering an overview of the ways to utilize persistent memory in the Java environment. In my nearly two years here at SNIA, this has been one of the most-requested topics. Steve Dohrmann and Soji Denloye are two of the brightest minds in enabling persistence, and this is sure to be an insightful presentation.

Persistent memory application capabilities are growing significantly.  Since the publication of the SNIA NVM Programming Model developed by the SNIA Persistent Memory Programming Technical Work Group, new language support seems to be happening every day.  Don’t miss the opportunity to see the growth of PM programming in such a crucial space as Java.

The presentation is on BrighTALK, and will be live on May 27th at 10am PST. You can see the details at this link.

Now I just have to find a clean cup.

This post is also cross-posted at the PIRL Blog.  PIRL is a joint effort by SNIA and UCSD’s Non-Volatile Systems Lab to advance the conversation on persistent memory programming.  Check out other entries here.

SNIA Exhibits at OCP Virtual Summit May 12-15, 2020 – SFF Standards featured in Thursday Sessions

Marty Foltyn Leave a comment

All SNIA members and colleagues are welcome to register to attend the free online Open Compute Project (OCP) Virtual Summit, May 12-15, 2020.

SNIA SFF standards will be represented in the following presentations (all times Pacific):

 Thursday, May 14:

The SNIA booth (link live at 9:00 am May 12) will be open from 9 am to 4 pm each day of OCP Summit, and feature Chat with SNIA Experts:  scheduled times where SNIA volunteer leadership will answer questions on SNIA technical work, SNIA education, standards, and adoption, and vision for 2020 and beyond.

The current schedule is below – and is continually being updated with more speakers and topics – so  be sure to bookmark this page.

And let us know if you want to add a topic to discuss!

Tuesday, May 12:

  • 10:00 am – 11:00 am – SNIA Education, Standards, and Technology Adoption, Erin Weiner, SNIA Membership Services Manager
  • 11:00 am – 12:00 pm – Uniting Compute, Memory, and Storage, Jenni Dietz, Intel/SNIA CMSI Co-Chair
  • 11:00 am – 12:00 pm – Computational Storage standards and direction, Scott Shadley, NGD Systems & Jason Molgaard, ARM, SNIA CS TWG Co-Chairs
  • 11:00 am – 12:00 pm – SNIA SwordfishTM and Redfish Standards and Adoption, Don Deel, NetApp, SNIA SMI Chair
  • 12:00 pm – 1:00 pm – SSDs and Form Factors, Cameron Brett, KIOXIA, SNIA SSD SIG Co-Chair
  • 12:00 pm – 1:00 pm – Persistent Memory Standards and Adoption, Jim Fister, SNIA Persistent Memory Enablement Director
  • 1:00 pm – 2:00 pm – SNIA Technical Activities and Direction, Mark Carlson, KIOXIA, SNIA Technical Council Co-Chair
  • 1:00 pm – 2:00 pm – SNIA Education, Standards, Promotion, Technology Adoption, Michael Meleedy, SNIA Business Operations Director

Wednesday, May 13:

  • 11:00 am – 12:00 pm – SNIA Education, Standards, and Technology Adoption, Arnold Jones, SNIA Technical Council Managing Director
  • 11:00 am – 12:00 pm – Persistent Memory Standards and Adoption, Jim Fister, SNIA Persistent Memory Enablement Director
  • 12:00 pm – 1:00 pm – Computational Storage Standards and Adoption, Scott Shadley, NGD Systems & Jason Molgaard, ARM, SNIA CS TWG Co-Chairs
  • 12:00 pm – 1:00 pm – SNIA Technical Activities and Direction, Bill Martin, Samsung, SNIA Technical Council Co-Chair
  • 12:00 pm – 1:00 pm – SSDs and Form Factors, Jonmichael Hands, Intel, SNIA SSD SIG Co-Chair
  • 1:00 pm – 2:00 pm – SNIA NVMe and NVMe-oF standards and direction, Mark Carlson, KIOXIA, SNIA Technical Council Co-Chair
  • 1:00 pm – 2:00 pm – SNIA SwordfishTM and Redfish standards and direction, Richelle Ahlvers, Broadcom, SNIA SSM TWG Chair, and Barry Kittner, Intel, SNIA SMI Marketing Chair

Thursday, May 14

  • 11:00 am – 12:00 pm – Uniting Compute, Memory, and Storage, Jenni Dietz, Intel, SNIA CMSI Co-Chair
  • 11:00 am – 12:00 pm – SNIA Education, Standards, and Technology Adoption, Erin Weiner, SNIA Membership Services Manager
  • 12:00 pm – 1:00 pm – SNIA SwordfishTM and Redfish standards activities and direction, Richelle Ahlvers, Broadcom, SNIA SSM TWG Chair, and Barry Kittner, Intel, SNIA SMI Marketing Chair
  • 12:00 pm – 1:00 pm – SNIA Technical Activities and Direction, Bill Martin, Samsung, SNIA Technical Council Co-Chair
  • 1:00 pm – 2:00 pm – Computational Storage Standards and Adoption, Scott Shadley, NGD Systems & Jason Molgaard, ARM, SNIA CS TWG Co-Chairs
  • 1:00 pm – 2:00 pm – SNIA NVMe and NVMe-oF standards activities and direction , Bill Martin, Samsung, SNIA Technical Council Co-Chair

Friday, May 15: 

  • 11:00 am – 12:00 pm – Computational Storage Standards and Adoption, Scott Shadley, NGD Systems & Jason Molgaard, ARM, SNIA CS TWG Co-Chairs
  • 11:00 am – 12:00 pm – SNIA SwordfishTM and Redfish standards and direction, Richelle Ahlvers, Broadcom, SNIA SSM TWG Chair
  • 12:00 pm– 1:00 pm – SNIA NVMe and NVMe-oF standards activities and direction, Mark Carlson, KIOXIA, SNIA Technical Council Co-Chair
  • 12:00 pm – 1:00 pm – SNIA Technical Activities and Direction, Bill Martin, Samsung, SNIA Technical Council Co-Chair
  • 1:00 pm – 2:00 pm – SNIA Education, Standards and Technology Adoption, Michael Meleedy, SNIA Business Services Director


Register today to attend the OCP Virtual Summit. Registration is free for all attendees and is open for everyone, not just those who were registered for the in-person Summit. The SNIA exhibit will be found here once the Summit is live.
 
Please note, the virtual summit will be a 3D environment that will be best experienced on a laptop or desktop computer, however a simplified mobile responsive version will also be available for attendees. No additional hardware, software or plugins are required.

Your Questions Answered on Persistent Memory Programming

Jim Fister Leave a comment

On April 14, the SNIA Compute Memory and Storage Initiative (CMSI) held a webcast asking the question – Do You Wanna Program Persistent Memory? We had some answers in the affirmative – answering the call of the NVDIMM Programming Challenge. 

The Challenge utilizes a set of systems SNIA provides for the development of applications that take advantage of persistent memory. These systems support persistent memory types that can utilize the SNIA Persistent Memory Programming Model, and that are also supported by the Persistent Memory Development Kit (PMDK) Libraries. 

The NVDIMM Programming Challenge seeks innovative applications and tools that showcase the features persistent memory will enable. Submissions are judged by a panel of SNIA leaders and individual contest sponsors.  Judging is scheduled at the convenience of the submitter and judges, and done via conference call.  The program or results should be able to be visually demonstrated using remote access to a PM-enabled server.

NVDIMM Programming Challenge participant Steve Heller from Chrysalis Software joined the webcast to discuss the Three Misses Hash Table, which uses persistent memory to store large amounts of data that greatly increases the speed of data access for programs that use it.  During the webcast a small number of questions came up that this blog answers, and we’ve also provided answers to subjects stimulated by our conversation. 

Q: What are the rules/conditions to access SNIA PM hardware test system to get hands on experience? What kind of PM hardware is there? Windows/Linux?

A: Persistent memory, such as NVDIMM or Intel Optane memory, enables many new capabilities in server systems.  The speed of storage in the memory tier is one example, as is the ability to hold and recover data over system or application resets.  The programming challenge is seeking innovative applications and tools that showcase the features persistent memory will enable.

The specific systems for the different challenges will vary depending on the focus.  The current system is built using NVDIMM-N.  Users are given their own Linux container with simple examples in a web-based interface.  The users can also work directly in the Linux shell if they are comfortable with it.

Q: During the presentation there was a discussion on why it was important to look for “corner cases” when developing programs using Persistent Memory instead of regular storage.  Would you elaborate on this?

A: As you can see in the chart at the top of the blog post, persistent memory significantly reduces the amount of time to access a piece of data in stored memory.  As such, the amount of time that the program normally takes to process the data becomes much more important.  Programs that are used to data retrieval taking a significant amount of time can then occasionally absorb a “processing” performance hit that an extended data sort might imply.  Simply porting a file system access to persistent memory could result in strange performance bottlenecks, and potentially introduce race conditions or strange bugs in the software.  The rewards of fixing these issues will be significant performance, as demonstrated in the webcast.

Q: Can you please comment on the scalability of your HashMap implementation, both on a single socket and across multiple sockets?

The implementation is single threaded. Multiple threading poses lots of overhead and opportunity for mistakes. It is easy to saturate performance that only persistent memory can provide. There is likely no benefit to the hash table in going multi-threaded. It is not impossible – one could do an example of a hash table per volume. I have run across multiple sockets that were slower with an 8% to 10% variation in performance in an earlier version.  There are potential cache pollution issues with going multi-threaded as well.

The existing implementation will scale one to 15 billion records, and we would see the same thing if we have enough storage. The implementation does not use much RAM if it does not cache the index.  It only uses 100mb of RAM for test data and does not use memory.

Q: How would you compare your approach to having smarter compilers that are address aware of “preferred” addresses to exploit faster memories?

The Three Misses implementation invented three new storage management algorithms.  I don’t believe that compilers can invent new storage algorithms.  Compilers are much improved since their beginnings 50+ years ago when you could not mix integers and floating-point numbers, but they cannot figure out how to minimize accesses.  Smart compilers will probably not help solve this specific problem.

The SNIA CMSI is continuing its efforts on persistent memory programming.  If you’re interested in learning more about persistent memory programming, you can contact us at pmhackathon@snia.org to get updates on existing contests or programming workshops.  Additionally, SNIA would be willing to work with you to host physical or virtual programming workshops.

Please view the webcast and contact us with any questions.

Feedback Needed on New Persistent Memory Performance White Paper

Marty Foltyn Leave a comment

A new SNIA Technical Work draft is now available for public review and comment – the SNIA Persistent Memory Performance Test Specification (PTS) White Paper.

A companion to the SNIA NVM Programming Model, the SNIA PM PTS White Paper (PM PTS WP) focuses on describing the relationship between traditional block IO NVMe SSD based storage and the migration to Persistent Memory block and byte addressable storage.  

The PM PTS WP reviews the history and need for storage performance benchmarking beginning with Hard Disk Drive corner case stress tests, the increasing gap between CPU/SW/HW Stack performance and storage performance, and the resulting need for faster storage tiers and storage products. 

The PM PTS WP discusses the introduction of NAND Flash SSD performance testing that incorporates pre-conditioning and steady state measurement (as described in the SNIA Solid State Storage PTS), the effects of – and need for testing using – Real World Workloads on Datacenter Storage (as described in the SNIA Real World Storage Workload PTS for Datacenter Storage), the development of the NVM Programming model, the introduction of PM storage and the need for a Persistent Memory PTS.

The PM PTS focuses on the characterization, optimization, and test of persistent memory storage architectures – including 3D XPoint, NVDIMM-N/P, DRAM, Phase Change Memory, MRAM, ReRAM, STRAM, and others – using both synthetic and real-world workloads. It includes test settings, metrics, methodologies, benchmarks, and reference options to provide reliable and repeatable test results. Future tests would use the framework established in the first tests.

The SNIA PM PTS White Paper targets storage professionals involved with: 

  1. Traditional NAND Flash based SSD storage over the PCIe bus;
  2. PM storage utilizing PM aware drivers that convert block IO access to loads and stores; and
  3. Direct In-memory storage and applications that take full advantage of the speed and persistence of PM storage and technologies. 

The PM PTS WP discussion on the differences between byte and block addressable storage is intended to help professionals optimize application and storage technologies and to help storage professionals understand the market and technical roadmap for PM storage.

Eden Kim, chair of the SNIA Solid State Storage TWG and a co-author, explained that SNIA is seeking comment from Cloud Infrastructure, IT, and Data Center professionals looking to balance server and application loads, integrate PM storage for in-memory applications, and understand how response time and latency spikes are being influenced by applications, storage and the SW/HW stack. 

The SNIA Solid State Storage Technical Work Group (TWG) has published several papers on performance testing and real-world workloads, and the  SNIA PM PTS White Paper includes both synthetic and real world workload tests.  The authors are seeking comment from industry professionals, researchers, academics and other interested parties on the PM PTS WP and anyone interested to participate in development of the PM PTS.

Use the SNIA Feedback Portal to submit your comments.

Share Your Experiences in Programming PM!

Jim Fister Leave a comment

by Jim Fister, SNIA Director of Persistent Memory Enabling

Last year, the University of California San Diego (UCSD) Non-Volatile Systems Lab (NVSL) teamed with the Storage Networking Industry Association (SNIA) to launch a new conference, Persistent Programming In Real Life (PIRL). While not an effort to set the record for acronyms in a conference announcement, we did consider it a side-goal.  The PIRL conference was focused on gathering a group of developers and architects for persistent memory to discuss real-world results. We wanted to know what worked and what didn’t, what was hard and what was easy, and how we could help more developers move forward.

You don’t need another pep talk about how the world has changed and all the things you need to do (though staying home and washing your hands is a pretty good idea right now).  But if you’d like a pep talk on sharing your experiences with persistent memory programming, then consider this just what you need.

We believe that continuing the spirit of PIRL — discussing the results of persistent memory programming in real life — should continue.

If you’re not aware, SNIA has been delivering some very popular webcasts on persistent programming, cloud storage, and a variety of other topics.  SNIA has a great new webcast featuring PIRL alumni Steve Heller, SNIA CMSI co-chair Alex McDonald, and me on the SNIA NVDIMM programming challenge and the winning entry. You can find more information and check the on-demand viewing at https://www.brighttalk.com/webcast/663/389451.

We would like to highlight more “In Real Life” topics via our SNIA webcast channel.  Therefore, SNIA and UCSD NVSL have teamed up to create a submission portal for anyone interested in discussing their real-world persistent memory experiences.  You can submit a topic here  https://docs.google.com/forms/d/e/1FAIpQLSe_Ypo_sf1xxFcPD1F7se02jOWrdslosUnvwyS0RwcQpWAHiA/viewform where we will evaluate your submission.  Acceptable submissions will be featured in conjunction with the SNIA channel over the coming months.

As a final note, this year’s PIRL conference is currently scheduled for July.  Even though most software developers are already used to social isolation and distancing from their peers, our organizing team has kept abreast of all the latest information to make a decision on the capability to do an in-person conference on that date.  In our last meeting, we agreed that it would not be prudent to hold the conference on the July date, and have tentatively rescheduled the in-person conference to October 13-14 of 2020. We will announce an exact date and our criteria for moving forward on that date in the coming weeks, so stay tuned!

Up to the Challenge!

Marty Foltyn Leave a comment

The SNIA Persistent Memory and NVDIMM Special Interest Group announced a programming challenge for NVDIMM-based systems in Q4 of 2019.  Participants get free online access to persistent memory systems based at the SNIA Technology Center using NVDIMM-Ns provided by SIG members AgigA Tech, Intel, SMART Modular, and Supermicro.  The goal of the challenge is to spark interest by developers in this new technology so they can understand more clearly how persistent memory applications can be developed and applied in 2020 environments and beyond.

Response to the NVDIMM Programming Challenge has been very positive.  Entrants to date have backgrounds from no experience programming persistent memory to those who develop persistent memory applications as part of their day jobs.

Read More

Judging Has Begun – Submit Your Entry for the NVDIMM Programming Challenge!

Marty Foltyn Leave a comment

We’re 11 months in to the Persistent Memory Hackathon program, and over 150 software developers have taken the tutorial and tried their hand at programming to persistent memory systems.   AgigA Tech, Intel,  SMART Modular, and Supermicro, members of the SNIA Persistent Memory and NVDIMM SIG, have now placed persistent memory systems with NVDIMM-Ns into the SNIA Technology Center as the backbone of the first SNIA NVDIMM Programming Challenge.

Interested in participating?  Send an email to PMhackathon@snia.org to get your credentials.  And do so quickly, as the first round of review for the SNIA NVDIMM Programming Challenge is now open.  Read More

Show Your Persistent Stuff – and Win!

Marty Foltyn Leave a comment

Persistent Memory software development has been a source of server development innovation for the last couple years.  The availability of the open source PMDK libraries (http://pmem.io/pmdk/) has provided a common interface for developing across PM types as well as server architectures.  Innovation beyond PMDK also continues to grow, as more experimentation yields open and closed source products and tools.

Read More