Category: Solid State Storage

Enterprise and Data Center Form Factor (EDSFF) technologies have come a long way since our 2020 SNIA CMSI webinar on the topic.  While that webinar still provides an outstanding framework for understanding – and SNIA’s popular SSD Form Factors page gives the latest on the E1 and E3 specifications – SNIA Solid State Drive Special Interest Group co-chairs Cameron Brett and Jonmichael Hands joined to provide the latest updates at our live webcast: EDSFF Taking Center Stage in the Data Center.  We had some great questions from our live audience, so our experts have taken the time to answer them in this this blog.

Q: What does the EDSFF roadmap look like? When will we see PCIe^® Gen5 NVMe™, 1.2, 2.0 CXL ™ cx devices?

As the form factors come out into the market, we anticipate that there will be feature updates and smaller additions to the existing specifications like SFF TA 1008 and SFF TA 1023.  There may also be changes around defining LEDs and stack updates.  The EDSFF specifications, however, are mature and we have seen validation and support on the connector and how it works at higher interface speeds. You now have platforms, backplanes, and chassis to support these form factors in the marketplace.  Going forward, we may see integration with other device types like GPUs, support of new platforms, and alignment with PCIe Gen 5.  Regarding CXL, we see the buzz, and having this form factor be the kind of vehicle for CXL will have a huge momentum. 

Q:  I’m looking for thoughts on recent comments I read about PCIe5 NVMe drives likely needing/benefitting from larger form-factors (like 25mm wide vs 22) for cooling considerations. With mass market price optimizations, what is the likelihood that client compute will need to transition away from existing M.2 (esp 2280) form factors in the coming years and will that be a shared form-factor shared with server compute (as has been the case with 5.25″,3.5″,2.5″ drives)?

We are big fans of EDSFF being placed on reference platforms for OEMs and motherboard makers. Enterprise storage support would be advantageous on the desktop.  At the recent OCP Global Summit, there was discussion on Gen 5 specifications and M.2 and U.2. With the increased demands for power and bandwidth, we think if you want more performance you will need to move to a different form factor, and EDSFF makes sense. 

Q:  On E1.S vs E3.S market dominance, can you refer to their support on dual-port modules? Some traditional storage server designs favor E3.S because of the dual port configuration. More modern storage designs do not rely on dual port modules, and therefore prefer E1.S. Do you agree to this correlation ? How will this affect the predictions on market share?

A:  There is some confusion about the specification support versus what vendors support and what customers are demanding.  The EDSFF specifications share a common pin out and connection specifications.  If a manufacturer wishes to support the dual port functionality, they can do so now.  Hyperscalers are now using E1.S in compute designs and may use E3 for their high availability enterprise storage requirements.  Our webcast showed the forecast from Forward Insights on larger shipments of E3 further out in time, reflecting the transition away from 2.5-inch to E3 as server and storage OEMs transition their backplanes.

Q:  Have you investigated enabling conduction cooling of E1.S and E3.S to a water cooled cold plate? If not, is it of interest?

OCP Global Summit featured a presentation from Intel about immersion cooling with a focus on the sustainability aspect as you can get your power usage effectiveness (PUE) down further by eliminating the fans in system design while increasing cooling.  There doesn’t seem to be anything eliminating the use of EDSFF drives for immersion cooling. New CPUs have heat pipes, and new OEM designs have up to 36 drives in a 2U chassis.  How do you cool that?  Many folks are talking about cooling in the data center, and we’ll just need to wait to see what happens!

Thanks again for your interest in SNIA and Enterprise and Data Center SSD Form Factors.  We invite you to visit our SSD Form Factor page where we have videos, white papers, and charts explaining the many different SSD sizes and formats in a variety of form factors. You may also wish to check out a recent article from Storage Review which discusses an E3.S implementation.

In our recent webcast Is the Data Really Gone? A Primer on the Sanitization of Storage Devices, our presenters Jonmichael Hands (Chia Network), Jim Hatfield (Seagate), and John Geldman (KIOXIA) took an in-depth look at exactly what sanitization is, what the standards are, and where sanitization is being practiced today.  If you missed it, you can watch on-demand their recommendations for the verification of sanitization to ensure that devices are meeting stringent requirements – and access the presentation slides at the SNIA Educational Library.  Here, in our Q&A blog, our experts answer more of your questions on data sanitization.

Is Over Provisioning part of the spare blocks or separate?

The main intent of an overprovisioning strategy is to resolve the asymmetric NAND behaviors of Block Erase (e.g., MBs) and Page Write (e.g., KBs) that allows efficient use of a NAND die’s endurance capability, in other words, it is a store-over capability that is regularly used leaving older versions of a Logical Block Addressing (LBA) in media until it is appropriate to garbage collect.

Spares are a subset of overprovisioning and a spare block strategy is different than an overprovisioning strategy. The main intent of a spare strategy is a failover capability mainly used on some kind of failure (this can be a temporary vibration issue on a hard disk drive or a bad sector).

The National Institute of Standards and Technology (NIST) mentions the NVMe^® Format with Secure Erase Settings to 1 for User Data erase or 2 for Crypto as a purge method. From what I can gather the sanitize was more a fallout of the format rather than anything that was designed. With the NVMe sanitize would you expect the Format with the Data Erasure options to be depreciated or moved back to a clear?

The Format NVM command does have a crypto erase, but it is entirely unspecified, vendor specific, and without any requirements. It is not to be trusted. Sanitize, however, can be trusted, has specific TESTABLE requirements, and is sanctioned by IEEE 2883.

The Format NVM command was silent on some requirements that are explicit in both NVMe Sanitize commands and IEEE 2883. It was possible, but not required for a NVME Format with Secure Erase Settings set to Crypto to also purge other internal buffers. Such behavior beyond the specification is vendor specific. Without assurance from the vendor, be wary of assuming the vendor made additional design efforts. The NVMe Sanitize command does meet the requirements of purge as defined in IEEE 2883.

My question is around logical (file-level, OS/Filesystem, Logical volumes, not able to apply to physical DDMs): What can be done at the technical level and to what degree that it is beyond what modern arrays can do (e.g., too many logical layers) and thus, that falls under procedural controls. Can you comment on regulatory alignment with technical (or procedural) acceptable practices?

The IEEE Security in Storage Working Group (SISWG) has not had participation by subject matter experts for this, and therefore has not made any requirements or recommendations, and acceptable practices. Should such experts participate, we can consider requirements and recommendations and acceptable practices.

Full verification is very expensive especially if you are doing lots of drives simultaneously. Why can’t you seed like you could do for crypto, verify the seeding is gone, and then do representative sampling?

The problem with seeding before crypto erase is that you don’t know the before and after data to actually compare with. Reading after crypto erase returns garbage…. but you don’t know if it is the right garbage.  In addition, in some implementations, doing a crypto erase also destroys the CRC/EDC/ECC information making the data unreadable after crypto erase.

Seeding is not a common defined term. If what was intended by seeding was writing known values into known locations, be aware that there are multiple problems with that process. Consider an Overwrite Sanitize operation. Such an operation writes the same pattern into every accessible and non-accessible block. That means that the device is completely written with no free media (even the overprovisioning has that pattern). For SSDs, a new write into that device has to erase data before it can be re-written. This lack of overprovisioned data in SSDs results in artificial accelerated endurance issues.

A common solution implemented by multiple companies is to de-allocate after sanitization. After a de-allocation, a logical block address will not access physical media until that logical block address is written by the host. This means that even if known data was written before sanitize, and if the sanitize did not do its job, then the read-back will not return the data from the physical media that used to be allocated to that address (i.e., that physical block is de-allocated) so the intended test will not be effective.

Are there other problems with Sanitize?

Another problem with Sanitize is that internal protection information (e.g., CRC data, Integrity Check data, and Error Correction Code data) have also been neutralized until that block is written again with new data. Most SSDs are designed to never return bad data (e.g., data that fails Integrity Checks) as a protection and reliability feature.

What are some solutions for Data Sanitization?

One solution that has been designed into NVMe is for the vendor to support a full overwrite of media after a crypto erase or a block erase sanitize operation. Note that such an overwrite has unpopular side-effects as the overwrite:

1. changes any result of the actual sanitize operation;
2. may take a significant time (e.g., multiple days); and
3. still requires a full-deallocation by the host to make the device useful again.

A unique complication for a Block Erase sanitization operation that leaves NAND in an erased state is not stable at the NAND layer, so a full write of deallocated media can be scheduled to be done over time, or the device can be designed to complete an overwrite before the sanitize operation returns a completion. In any/either case, the media remains deallocated until the blocks are written by the host.

Can you kindly clarify DEALLOCATE all storage before leaving sanitize ? What does that mean physically?

Deallocation (by itself) is not acceptable for sanitization. It is allowable AFTER a proper and thorough sanitization has taken place. Also, in some implementations, reading a deallocated logical block results in a read error. Deallocation must be USED WITH CAUTION. There are many knobs and switches to set to do it right.

Deallocation means removing the internal addressing that mapped a logical block to a physical block. After deallocation, media is not accessed so the read of a logical block address provides no help in determining if the media was actually sanitized or not. Deallocation gives as factory-fresh out of the box performance as is possible.