Facebook: How do we build information on AI and VR hardware-
Facebook,AI,VR-IT
Facebook is creating the next generation of open source hardware, integrating new technologies into their data center platform, Facebook uses a lot of powerful technologies--including solid state drives (SSDs), graphics processing units (GPUs), non-volatile memory (NVM) and Flash memory clusters (JBOFs)-to create a new server and storage devices, and accelerate the infrastructures.
The upgrade is part of Facebook to create powerful data center network, the network will expand the boundaries of Internet services.
"For more than 10 years, our experience will be more dependent on artificial intelligence and virtual reality technologies," Facebook CEO Mark Zuckerberg said. "It put forward higher requirements for performance, by open computing project (Open Compute Project, hereafter referred to as OCP), we are developing a global infrastructure, so that everyone can enjoy Internet services. ”
Facebook open computing Summit (Open Compute Summit, hereinafter referred to as OCS) posted on the progress of the project. Members of OCS is growing open source hardware community gatherings, they designed the product rejuvenation in the Facebook data server. Summit shows a lot of infrastructure upgrades. Included:
Rebuild the server scales form under the same quantity of energy consumption to increase performance.
New high performance data analysis server, powered by GPUs, not CPUs.
Improved separated storage devices (storage sled) where first JBOD (just a bunch of disks) become much faster JBOF (just a bunch of Flash).
Improvement of non-volatile memory (NVM) experiments, providing more choices for Storage Tiering.
The Summit marked the five years of the OCP project also helps people reflect on 2011 innovation on Facebook to promote its release since the project was set up, developments in the OCP.
"We should pay attention to the progress made," OCP said Jason Taylor, Chairman of the project, he and one of the Facebook Vice President of Infrastructure. "OCP is one for the engineers to get together to create amazing products. ”
"I feel a great deal of power because we are beyond Super host (hyperscale), also entered the financial and telecommunications sectors. "He said.
Server: next-generation designs
Facebook revolutionized the design and infrastructure servers, from the traditional two-server processors into a single Intel-based Xeon-D processor system-on-chip (SoC), the processor consumes less power, also resolved several architectural issues.
The Mono Lacke Server Board into an Assembly called Yosemite, Yosemite each unit can be installed on the chassis slide 4 SoC, or 4 Mono Lake. Facebook engineer Vijay Rao and Edwin Smith on the Facebook engineering blog describes this new design.
"We worked closely with Intel on a new processor design, and redesign their server infrastructure to build a system that meets our needs, also widely used for other companies. "They wrote in a blog post. "The results show that, with more low-power CPUs, a single processor server perform better than the dual-processor servers in the network load, are better suited for data center load ... Meanwhile, we redesigned the server infrastructure, on the basis of the same power, hold each rack-double the number of CPUs. ”
New design improved the processor and the processor, communication between the processor and memory efficiency.
"We the CPU was simplified to just meet our requirements," the two engineers said, "we removed a QPI link, cut the cost of Intel. Taking into account our server will be based on the same slot, it also solves the problem we NUMA. We design for system-on-chip integrated chip set, much simpler. This interface consistent CPU thermal design power (TDP) lower. Meanwhile, we redesigned the server infrastructure, under the same conditions of power supply, hold each rack-double the number of CPUs. ”
This allows Facebook to create a higher performance can be obtained on each rack, server architecture, while ensuring that each cabinet energy consumption density remain below 11-kilowatt.
Powerful artificial intelligence data analysis server
Progress in the latest GPUs are sharing on Facebook, in recent years, the GPU has played an important role in the high-performance computing. GPUs are initially used to enhance desktop PC performance, graphics operations, but now I help some of the world′s most powerful supercomputers.
Facebook using GPUs to artificial intelligence and machine learning data analysis with more computing power. The artificial intelligence laboratory of neural networks to solve new problems. This requires strong computing power.
"We have invested a lot in artificial intelligence. "Facebook engineering and infrastructure (Engineering and Infrastructure) sector globally and Jay Parikh said," at present, the artificial intelligence applied to the Facebook Newsfeed, and helps us provide better service. Can also help users more safe use of Facebook in their daily lives. ”
Big Sur system uses accelerated NVIDIA Tesla computing platforms, with 8 high performance GPUs, each up to 300 Watts of power, flexible configuration across multiple PCI-e connection. Facebook optimized these new heat and power consumption of the server control so that it can work with the standard CPU servers in the data center.
Big Sur systems to optimize performance and response time helping Facebook handle more data, which greatly shortens the time of the trained neural network.
"The performance has been greatly improved," Parikh said, "we in a few months the deployment of thousands of the machines. Allows us to have the ability to apply this technology to more of our product instances. ”
Storage: Just a Buch of Flash
For many years, Facebook has used Flash memory to accelerate Server boot drive to improve caching efficiency. With infrastructure to further expand the scale of the company, Facebook has created a new type of "modules", consolidating more Flash in Server operations. Facebook has improved its early open computer--called the Knox--storage drives, using solid state drives (SSDs) to replace storage hard drive storage (HDDs), storage units from the disk clusters (JBOD) transferred to the Flash memory clusters (JBOF).
Clusters of Facebook and Intel to develop this new type of Flash unit, known as the Lighting, reaction speed using NVMe, NVMe high-speed PCI bus interface is optimized for SSDs. Under our Parikh displayed on open computing Summit slides and see specific parameters.
As a decentralized (disaggregated) storage devices, Lightning supports a variety of different applications. "This gives the application we are developing a new construction of high performance memory modules. "Parikh said. Parikh said there will be more innovation in storage, especially the innovative use of non-volatile memory (NVM).
"In the storage industry, also continued to expand the disk drive, but did not get better reliability and response time and IOPS (per second read and write times) has never been raised. "Parikh said," Flash small progress, but durability has not greatly improved. We fall into a pattern: continue to expand the scale, but from a performance point of view, we do not really need. ”
Facebook in the application of new types of NVM, Intel and Micron jointly developed 3D XPoint technologies in particular, saw the potential answers. Parikh calls for open computing community to focus on the technology, or solve storage challenges to industry.
Parikh said, we can (based on these choices, different technologies such as NVM,Flash), start to think about storage issues, classified storage requirements (in terms of performance, capacity) layout, give us more lever of price and performance, expansion of performance or storage, or to optimize the cost. NVM he said was between DRAM and NAND a concern.
Facebook is MyRocks open source project test NVM configuration, the project is based on RocksDB and MYSQL database technology.
The road is long: to extend the flood of data
Facebook to spare no effort to facilitate faster, more infrastructure because of the strong subscriber growth, at present, Facebook users reached 1.06 billion, WhatsApp 1 billion, Facebook Messenger 800 million, Instagram 400 million. Mr Zuckerberg′s ambition in the field of virtual reality is one of the drivers, and Facebook as evidenced by the $ 2 billion acquisition of virtual reality pioneer Oculus.
Virtual reality provides an immersive 3D experience, many analysts say, the golden period of virtual reality is coming. Zuckerberg believes Facebook could give its social network virtual reality experience.
"Soon, we will live in a place where everybody can feel like sharing and experiencing all the scenes of the world," Zuckerberg during the recent Mobile World Congress (MWC) said. "Just imagine, you can at any time you can sit in the bonfires, and stayed with friends; or if at any time you can with friends in a private cinema. Imagine organized in any corner of the globe for a meeting or Conference. All of this will become a reality, which is why Facebook early investing heavily in virtual reality, we really want to provide the social experience. ”
This extremely high demands on the infrastructure, full virtual reality video HD video size is now more than 20 times times.
"Large files or to be high quality and obstacles to promote 360 videos or virtual reality on a large scale. "Facebook Evgeny Kuzakov and David Pio writes, they outlined recent advances in decoding and compression technology of virtual reality files. Facebook layout is equidistant cylindrical projection on a 360 video (equirectangular layouts) moved to use the cube layout (cube format), file size reduced by 25%.
However, Facebook realized that virtual reality video live push faster Internet, Facebook cannot achieve. So the Facebook style of open computing model, Telecom Infra Project project is proposed, with Equinix, Intel, Nokia, SK Telecom, t-mobile and Germany telecoms joint development 5G network, accelerating global network development.
"In order to meet the global (video and virtual reality) the data challenges, upgrading traditional telecom infrastructure than required," Parikh said, "to meet the challenges of these new technologies, find new opportunities, accelerate the pace of innovation in telecommunications infrastructure, it is necessary. ”
Facebook：我们如何打造AI和VR硬件 -
Facebook,AI,VR - IT资讯
Facebook正在打造下一代开源硬件，将新技术融入其数据中心平台，Facebook使用大量强大科技——包括固态硬盘（SSDs）,图形处理器（GPUs）,非易失性存储器（NVM）和闪存簇（JBOFs）——以打造新的服务器和存储装置，加速基础设施。
这些升级是Facebook打造强大数据中心网络的一部分，这一网络将扩展互联网服务的边界。
“接下来的十几年，我们的体验将更加依赖于人工智能和虚拟现实这类技术，”Facebook CEO马克·扎克伯格说。“这对计算性能提出了更高要求，通过开放计算项目（Open Compute Project，以下简称OCP），我们正在开发全球性的基础设施，让人人都能享受互联网服务。”
Facebook在开放计算峰会（Open Compute Summit，以下简称OCS）上公布了项目进展。OCS是不断扩大的开源硬件社区成员的聚会，他们设计的产品在Facebook的数据服务器上焕发活力。峰会展示了许多基础设施的升级。包括：
重新打造的服务器尺度形态来在同等能源消耗数量下提升性能。
新的高性能数据分析服务器，由GPUs驱动，而非CPUs。
改进后的存储设备分隔单位(storage sled），在这里，最初的JBOD（just a bunch of disks）变成快得多的JBOF（just a bunch of flash）。
改进非易失性存储器（NVM）实验，为存储分层提供更多选择。
此次峰会标志着OCP项目成立五周年，也促使人们反思自2011年Facebook为推动其发布的设计创新而成立该项目以来，OCP的发展情况。
“我们现在取得的进步是值得关注的，”OCP项目董事长Jason Taylor表示，他同时也是Facebook Infrastructure部门副总裁之一。“OCP就是一个让工程师聚在一起，创造惊艳产品的地方。”
“我感受到了巨大的动力，因为我们超越了超大型主机（hyperscale），还进入了金融和电信领域。”他说。
服务器：下一代设计
Facebook彻底革新了服务器设计和基础设施，从传统的双服务器处理器转为基于单个英特尔Xeon-D处理器系统芯片（SoC），该处理器耗能更少，还解决了几个架构问题。
The Mono Lacke服务器的主板装入到一个叫Yosemite的组装单位中，Yosemite的每一个滑动单元底架上可以安装4个SoC，即4个Mono Lake。Facebook工程师Vijay Rao和Edwin Smith在Facebook工程博客上描述了这一新设计。
“我们就新处理器的设计与英特尔紧密合作，同时重新设计自家的服务器基础设施以打造一个满足我们需求的系统，也能为行业其他公司所广泛采用。”他们在博客中写道。“结果表明，配备更低功耗的CPUs的单处理器服务器在网络负载上比双处理器服务器表现更好，也更适合数据中心负载…与此同时，我们重新设计的服务器基础设施，在同等电源基础上，容纳每个机架双倍的CPUs数量。”
新的设计提升了处理器和处理器之间、处理器和内存之间的通信效率。
“我们将CPU简化至正好能满足我们要求，”这两位工程师表示，“我们移除了QPI链接，缩减了英特尔的成本。考虑到我们的服务器将基于同一的插槽，这也解决了我们NUMA问题。我们将其设计为整合芯片集的系统芯片，更加简单。这种接口一致的CPU散热设计功耗（TDP）更低。与此同时，我们重新设计的服务器基础设施，在同等电源条件下，容纳每个机架双倍的CPUs数量。”
这使得Facebook制造出一个可以在每个机架上获得更高性能的服务器体系结构，同时确保每个机柜能量消耗密度保持在11千瓦以下。
强大的人工智能数据分析服务器
Facebook在会上还分享了GPUs最新进展，近年来，GPU在高性能计算中发挥了重要作用。GPUs最初是用于提升桌面PC性能，处理图形运算，但是现在却帮助提升全球一些性能最强大的超级计算机。
Facebook使用GPUs给人工智能和机器学习数据分析带来更强大的计算能力。其人工智能实验室训练神经网络去解决新问题。这就需要强大的计算能力。
“我们已经在人工智能技术方面投入了很多。”Facebook工程和基础设施（Engineering and Infrastructure）部门全球老大Jay Parikh说，“目前，人工智能应用于Facebook的Newsfeed，帮助我们提供更好的广告服务。也能帮助用户在日常生活中更安全地使用Facebook。”
Big Sur系统使用了英伟达Tesla加速计算平台，拥有8个高性能GPUs，每个功率达300瓦特，在多个PCI-e连接之间灵活配置。Facebook已经优化了这些新型服务器的发热和功耗控制，使其可以与数据中心中配备标准CPU的服务器协同工作。
Big Sur系统优化了性能和反应时间，帮助Facebook处理更多的数据，极大地缩短了其训练神经网络的时间。
“性能有了极大的提升，”Parikh说，“我们在数月间就部署了数千台这种机器。使得我们有能力将该技术应用于公司更多的产品实例。”
存储：Just a buch of Flash
多年来，Facebook已经在用闪存来加速服务器引导驱动，提升缓存效率。随着公司基础设备进一步扩大规模，Facebook创造了一种新型“模块”，在服务器运算中整合更多的闪存。Facebook已经改进了初代开放计算机——即Knox——的存储驱动器，使用固态硬盘（SSDs）取代存储硬盘存储（HDDs），存储单元从磁盘簇（JBOD）转到了闪存簇（JBOF）。
Facebook和英特尔合作开发这种新型的闪存簇单元，称之为Lighting，反应了使用NVMe带来了速度提升，NVMe是为SSDs优化的高速PCI总线接口。下面我们通过Parikh在开放计算峰会上展示的幻灯片来看看具体的参数。
作为一个分散式（disaggregated）存储设备，Lightning支持各种不同的应用。“这为我们正在开发的应用带来了一个高性能存储式的新型建造模块。”Parikh说。Parikh表示将来会有更多存储方面的创新，尤其是创新地使用非易失性存储器（NVM）。
“在存储行业，磁盘驱动器还在持续扩大，但是可靠性和反应时间并未变得更好，IOPS（每秒读写次数）也没有提升。”Parikh表示，“闪存小有进步，但是耐久性也没有极大地提升。我们陷入这样一个模式：规模在不断扩大，但是从性能的角度看，我们并未获得真正需要的。”
Facebook在新型的NVM应用方面，尤其是英特尔和镁光共同开发的3D XPoint技术，看到了潜在答案。Parikh呼吁开放计算社区将重点放在该技术上，或能解决存储行业面临的挑战。
Parikh说，我们可以（基于这些选择，指NVM，Flash等不同技术）开始思考我们的存储问题，把存储需求进行分级（按照性能，容量）布局，带给我们更多的价格和性能杠杆，实现性能或者储能的拓展，或者对成本进行优化。他表示NVM是介于DRAM和NAND之间一个值得关注的。
Facebook正在MyRocks开源项目测试NVM配置，该项目是基于MYSQL和RocksDB数据库技术之上。
前路漫漫：为到来的数据洪流进行扩展
Facebook不遗余力地推动更快、性能更强的基础设施建设的原因在于用户增长，目前，Facebook用户数达到10.6亿、WhatsApp 10亿、Facebook Messenger 8亿、Instagram 4亿。扎克伯格在虚拟现实领域的野心也是推动因素之一，Facebook 20亿美元收购虚拟现实先驱Oculus就是明证。
虚拟现实可提供沉浸式3D体验，很多分析人士认为，虚拟现实技术的黄金期即将到来。扎克伯格相信Facebook可赋予其社交网络虚拟现实体验。
“很快，我们将生活在一个人人都可以身临其境般分享和体验全部场景的世界，”扎克伯格在近期的世界移动通信大会（MWC）上说。“想象一下，你可以在任何时候你都可以坐在篝火前，与朋友待在一起；或者在任何时间都可以与朋友们在私人电影院看电影。想象下在全球任意一个角落开会或举办大会。所有这些都将成为现实，这也是为什么Facebook这么早在大力投资虚拟现实，我们真的很希望提供这些社交体验。”
这对基础设施建设提出了极高的要求，完整的虚拟现实视频容量大小是现在的高清视频的20倍之多。
“超大的文件或成为高质量、大规模地推广360视频或虚拟现实的障碍。”Facebook的Evgeny Kuzakov and David Pio写道，他们近期概述了虚拟现实文件的解码和压缩技术的进展。Facebook在360视频上正从等距柱状投影布局（equirectangular layouts）迁入使用立方体版式（cube format )，文件大小减少了25%。
但是，Facebook意识到虚拟现实视频实时推送需要更快的网络，Facebook无法独立实现。所以Facebook仿照开放计算的模式，提出了Telecom Infra Project项目，与Equinix、英特尔、诺基亚、SK电讯、T-Mobile和德国电信共同开发5G网络，加速全球网络发展。
“为了迎接全球（视频和虚拟现实）的数据挑战，传统电信基础设施的升级不及人们所需，”Parikh表示，“对迎接这些新的技术挑战、发现新的机遇来说，加快电信基础设施的创新步伐，很有必要。”