Author: Marty Foltyn

Judging Has Begun – Submit Your Entry for the NVDIMM Programming Challenge!

Marty Foltyn Leave a comment

We’re 11 months in to the Persistent Memory Hackathon program, and over 150 software developers have taken the tutorial and tried their hand at programming to persistent memory systems.   AgigA Tech, Intel,  SMART Modular, and Supermicro, members of the SNIA Persistent Memory and NVDIMM SIG, have now placed persistent memory systems with NVDIMM-Ns into the SNIA Technology Center as the backbone of the first SNIA NVDIMM Programming Challenge.

Interested in participating?  Send an email to PMhackathon@snia.org to get your credentials.  And do so quickly, as the first round of review for the SNIA NVDIMM Programming Challenge is now open.  Read More

Show Your Persistent Stuff – and Win!

Marty Foltyn Leave a comment

Persistent Memory software development has been a source of server development innovation for the last couple years.  The availability of the open source PMDK libraries (http://pmem.io/pmdk/) has provided a common interface for developing across PM types as well as server architectures.  Innovation beyond PMDK also continues to grow, as more experimentation yields open and closed source products and tools.

Read More

It’s a Wrap for SNIA and the Solid State Storage Initiative at Flash Memory Summit 2019

Marty Foltyn Leave a comment

A Best of Show award, over 12 hours of content, three days of demos, and a new program drawing attention to persistent memory programming completed – Flash Memory Summit 2019 is officially a success!

SNIA volunteers were again recognized for their hard work developing standards for datacenters and storage professionals with a “Most Innovative Flash Memory Technology” FMS Best of Show award. This year, it was SNIA’s Object Drive Technical Work Group who received kudos for the SNIA Technical Position Key Value Storage API Specification.  Jay Kramer, head of the FMS awards program, presented the award to Bill Martin, Chair of the Object Drive TWG, commenting “Key value store technology can enable NVM storage devices to map and store data more efficiently and with enhanced performance, which is of paramount significance to facilitate computational storage.  Flash Memory Summit is proud to recognize the SNIA Object Drive Technical Work Group (TWG) for creating the SNIA Technical Position Key Value Storage API Specification Version 1.0 defining an application programming interface (API) for key value storage devices and making this available to the public for download.

Read More

Your Questions Answered – Now You Can Be a Part of the Real World Workload Revolution!

Marty Foltyn Leave a comment

The SNIA Solid State Storage Initiative would like to thank everyone who attended our webcast: How To Be Part of the Real World Workload Revolution.  If you haven’t seen it yet, you can view the on demand version here.  You can find the slides here.

Eden Kim and Jim Fister led a discussion on the testmyworkload (TMW) tool and data repository, discussing how a collection of real-world workload data captures can revolutionize design and configuration of hardware, software and systems for the industry.   A new SNIA white paper available in both English and Chinese authored by Eden Kim, with an introduction by Tom Coughlin of Coughlin Associates and Jim Handy of Objective Analysis, discusses how we can all benefit by sharing traces of our digital workloads through the SNIA SSSI Real-World Workload Capture program.

Read More

SNIA at Flash Memory Summit 2019 – Your Guide Here!

Marty Foltyn Leave a comment

SNIA technical work and education advances will play a prominent role in the program at the 2019 Flash Memory Summit, August 5-8, 2019, in Santa Clara, CA.  Over 40 speakers will present on key standards activities and education initiatives, including the first ever FMS Persistent Memory Hackathon hosted by SNIA.  Check out your favorite technology (or all), and learn what SNIA is doing in these sessions:

Read More

Register for the PIRL Conference Today

Marty Foltyn Leave a comment

Registration is now open for the upcoming Persistent Programming in Real Life (PIRL) Conference – July 22-23, 2019 on the campus of the University of California San Diego (UCSD).

The 2019 PIRL event features a collaboration between UCSD Computer Science and Engineering, the Non-Volatile Systems Laboratory, and the SNIA to bring industry leaders in programming and developing persistent memory applications together for a two-day discussion on their experiences.

Read More

Calling All Real-World Workloads

Marty Foltyn Leave a comment

Video streaming is an easy-to-understand workload from the I/O perspective, right?  It’s pretty obvious that it’s a workload heavy on long, streaming reads. The application can be modeled with a consistent read flow, and the software tests should be easy.  However, an analysis of the real-world workload shows something very different. At the disk level, the reads turn out to be a rapid flow of 4k and 8k block reads from a solid-state-disk.  Further, other processes on the system also add in a small amount of 4k and 8k writes in the midst of the reads. All of this impacts the application –and an SSD — which was likely heavily tested on the basis of long, streaming reads.

Understanding the real-world characteristics of a workload can be a significant advantage in the development of new hardware, new systems, and new applications.   The SNIA Solid State Storage Initiative (SSSI) and SSSI member company Calypso Systems are providing an opportunity to build a repository of workloads for the industry to use for real-world testing, as outlined in a new SSSI white paper How to Be a Part of the Real-World Workload Revolution. This paper is also available in Chinese at the SSSI Knowledge Center White Papers page.

By going to the TestMyWorkload site, anyone can participate by providing a trace capture of an I/O workload that can be used by others to develop better products. The capture itself traces the block transfers, but does not capture actual data.  Any workload replay would use representative blocks, so there are no concerns about data security or integrity from these captures.

The repository can be used by any participant to test hardware and software, and can help system vendors and users optimize configurations for the best performance based on real-world data.  By participating in this effort, organizations and individuals can provide insight and gain from the knowledge of all the contributors.

Follow these three steps to be a part of the revolution today!

1.  Read the white paper.

2.  Download the free capture tools at TestMyWorkload.com.

3. Mark your calendar and register HERE to learn more in the free SNIA webcast How to Be a Part of the Real-World Workload Revolution on July 9 at 11:00 am Pacific/2:00 pm Eastern.

New Conference Seeking PIRLs of Wisdom

Marty Foltyn Leave a comment

UCSD Computer Science and Engineering, the Non-Volatile Systems Laboratory, and the Storage Networking Industry Association (SNIA) are inviting submissions of proposals for presentation at the first annual Persistent Programming in Real Life (PIRL) conference.  PIRL brings together software development leaders interested in learning about programming methodologies for persistent memories and sharing their experiences with others. This is a meeting for developer project leads on the front lines of persistent programming; not sales, marketing, or non-technical management.

PIRL is small, with attendance limited to under 100 people, including speakers.  It will discuss what real developers have done, and want to do, with persistent memory.  It will involve what worked, what didn’t, what was easy and hard, what was surprising, and what others can learn from the experience.  Presenters are encouraged, and even expected, to show and write code live in the presentation in a comfortable and dynamic peer environment.

Possibilities for presentations include, but are not limited to:

•  Experiences on a particular project

•  Live code development showing new concepts

•  Code challenges

•  New tools for programming

All attendees will be provided access to a development environment to respond to code challenges, or to show their own work in small forums.  This is intended to be a competition-free atmosphere for peers to network with each other to advance the use of persistent memory in the industry and academia.  By combining many of the industry leaders with the academic lights driving practical applications of new technology, peers at PIRL will encourage forward progress for adoption of persistent memory in the marketplace.

Keynote speakers include key personnel from Dreamworks, VMWare, Oracle, Eideticom, and Intel.

PIRL will be hosted by the Non-Volatile Systems Laboratory at the University of California, San Diego.  It will be held at Scripps Forum on July 22nd to 23rd, 2019, with optional events starting July 21st. Pre-registration will be $400.

We’re excited to present this new conference, and we’re excited for you to participate.  Submit your presentation or code challenge idea today. Submissions are due by Monday, June 10th.

 

Trends in Media and Entertainment Storage – Your Questions Answered from Our Webcast

Marty Foltyn Leave a comment

Thanks to all who attended or listened on-demand to our recent SNIA Solid State Storage Initiative (SSSI) webcast on Trends in Worldwide Media and Entertainment Storage. Motti Beck of Mellanox Technologies and Tom Coughlin, SSSI Education Chair and analyst with Coughlin Associates, got rave reviews for their analysis of this important market.  Feedback comments included “Good overview with enough details for me to learn something”; “Really appreciate the insight into the ME businesses”; and “Just in time for the upcoming NAB Show!”.  We appreciate your interest and enthusiasm!

Important to every SNIA webcast are the Questions – and we got quite a few on this one.  Thanks in advance to Tom Coughlin, who provided the answers below.  Send any more questions to us at asksssi@snia.org with the subject- M&E Webcast Questions.  Happy reading, and we hope to see you at one of our upcoming webcasts or events.

 

Q.  What is the best form to store(age) the format video, NAS or SAN?

A.  Well, it depends.  A SAN can directly access the data blocks that make up the video file, these can be quickly transported to the workstation.  There they are reassembled into the video file.  A properly configured SAN can provide faster access to data, particularly if many users are accessing the same data in the storage system.  SANs can be appropriate for a larger production facility.  A NAS may provide somewhat slower access, but provides individual access to individual files.  NAS storage can be an appropriate shared storage for smaller production facilities where there are fewer users or the users don’t access the same files at the same time.

 

Q.  Are there an M&E-specific performance benchmarks or other qualification tools recommended for storage subsystem selection?

A.  That is an interesting question.  I know about several general storage performance benchmarks, such as SPC (https://spcresults.org/benchmarks).  There are storage performance tests offered by some M&E industry suppliers, such as one from AJA System Test (https://www.aja.com/products/aja-system-test).  This is probably an area that could use some additional development.

 

Q.  Revenue share by type and use case – is this on the decline or rise?  What are the YoY trends?

A.  If I understand this right, you are asking questions about revenue growth for different media and entertainment use cases or different parts of the workflow on an annual basis.  That information is in the 2018 Digital Storage for Media and Entertainment Report (https://tomcoughlin.com/tech-papers/)

 

Q.  A question for Tom Coughlin.  You  said 66% will use private or public cloud for archiving in 2018.  Do you have the breakdown between the the two?

A.  It is a combination but given the concerns of the industry, I suspect this is mostly private cloud in 2018.

 

Q.  When Tom says “post production” storage, is that primary, secondary/nearline, or both?

A.  If I understand the question right, this is all storage used in post-production, which can use a primary and secondary storage tier, particularly in a larger facility which is economizing on its storage costs.

 

Q.  With regard to HDD storage, does the interface trend continue to be SATA/SAS?  Does the back end workload look to benefit from SMR or dual-actuator technology?

A.  For the time being HDDs will be SATA and SAS.  There are now some HDD storage systems with NVMe on the back end and it will be interesting to see how this develops.  I am sure that M&E users will benefit from SMR and dual-actuator HDDs.  SMR will be good for active archiving in particular and dual actuator will allow faster access to HDD data, a benefit for video projects.

 

Q.  Unless I missed it, you made no mention of software-defined storage as a viable method for storing the growing amount of data in M&E.  Was that taken into consideration when you did your survey?

A.  Software defined storage can be an important element in media and entertainment storage and is finding increasing use in this and other applications.

 

Q.  Is the cloud archive, primary copy or secondary (insurance policy with limited to no access)?

A.  It depends upon the organization, although I think for many studios and larger organizations, they may keep content on tape and even off-line tape as well.  Cloud archives do allow access to data, the usual issue is the cost of egressing that content.