Temel Bilgiler Genel Bakış

... konulu sunumlar: "Temel Bilgiler Genel Bakış"— Sunum transkripti:

1 Temel Bilgiler Genel Bakış
Tolga Şentekin Matris Bilişim Teknolojileri 2011 NetApp Confidential – Limited Use

2 NetApp Confidential – Limited Use
İlk Bakışta... Katmanlar WAFL, NVRAM, RAID-DP Donanım Platformlar, Disk Yapısı, Flash Cache Yazılım Data ONTAP, Standart Özellikler, Ek Özellikler NetApp Confidential – Limited Use

3 NetApp Technolojileri
NetApp Confidential – Limited Use

4 Foundation – NetApp Çekirdek Yapılar
WAFL NVRAM RAID-DP NetApp Confidential – Limited Use

5 Data ONTAP Bileşenleri: WAFL VS “Geleneksel” Sistemler
Geleneksel Dosya Sistemleri Datanın disk üzerindeki yeri Herhangi Biryer Diskin Genelinde raid sisteminin belirlediği yerde Metadata Disk üzerinde herhangi bir yerde (root inode Hariç) Belirli bölgelerde Varolan data / metadata güncelleme Boş Bloklara yazılır (Orjinaller kalır) Varolan datanın üzerine yazılır Dosya sistemi tutarlılığı Tasarımın temelinde, bozulma yok Tüm yazma işlemleri dikkatli yapılmalıdır Planlanmamış felaket Yeniden başladığı anda hazır Yavaş açılma sistem kontrolü zorunlu RAID ile uyum Full sektörlere blok yazabilme Disk üzerinde update için arama Snapshot kopyaları / versiyonları Tasarımın temelinde extra copy on write işlemi gerektirir Data ONTAP Components WAFL: WAFL vs. “Traditional” Filesystems data ontap components: WAFL versus “traditional” file systems WAFL (Write Anywhere File Layout) is the NetApp file system. It is the file system layer of Data ONTAP, but what does the name WAFL mean? Sometimes potential new customers are confused about the meaning. Sometimes this confusion has been planted by NetApp competitors — an insidious sales technique sometimes known as FUD — fear, uncertainty and doubt. Sometimes competitors will suggest that WAFL does not protect data stored on disk because WAFL will store the data on disk just “anywhere.” However, that is actually not what WAFL means. In fact, it is just the opposite. The important point is that unlike the vast majority of file systems that require metadata to be recorded to a particular physical location on the disk, the WAFL file system is able to write metadata anywhere on the disk. From a performance point of view, WAFL attempts to avoid the disk head having to write data in one location and then having to move up to a special portion of the disk to update the inodes — the metadata — and then move back to write some more data, and move again to update inodes, and so on across the physical disk medium. Head seeks happen very fast, but on server class systems you have thousands of disk accesses going on per second. This adds up very quickly and will greatly impact the performance of the system, particularly on write operations. WAFL does not have that handicap, and writes the metadata in line with the rest of the data. Write anywhere refers to the file system’s capability to write any class of data at any location on the disk; in other words, it can choose where to put the data. The basic goal of WAFL is to write to the first best available location. First best available is always going to be a complete stripe across an entire RAID group that utilizes the least amount of head movement to access. That is arguably the most important criteria for choosing where WAFL is going to locate data on a disk. That is what the term “write anywhere” refers to, the location of the metadata. Data ONTAP has control over where everything goes on the disks, so it can decide the most optimal location for data and metadata. This fact has very significant ramifications for the way Data ONTAP does everything, but particularly in the operation of RAID and the operation of Snapshot technology.

6 NVRAM İşlemi Client Storage System Dual-attached FC GbE
NVRAM operation nvram operation Looking at a basic setup of systems, we will step through the process that WAFL uses integrating the NVRAM into the system of reads and writes.

7 NVRAM Operation (Devam)
Client Operasyon pil destekli RAM üzerinde kontroller hatasından arınmış durumda Operation aynı zamanda işlemlerin devamı için ana hafızada Storage System Operation Main Memory NIC NIC N V R A M Main Memory ack + BATT - NIC = Network Interface Card NVRAM operation nvram operation (cont.) Let’s take a close look at NVRAM and WAFL integration. The controller contains a special chunk of RAM called NVRAM. In this case, NV means nonvolatile. It is nonvolatile because it has a battery, so if something happens like a sudden disaster striking the system, the janitor tripping over the power cords, or whatever, the data stored in NVRAM is not lost. Once data gets to a NetApp storage system, it is treated in the exactly the same way whether it came through a SAN or NAS connection. As I/O requests come into the system, they first go to RAM. The RAM on a NetApp system is used as in any other system; it is where Data ONTAP does active processing. As the write requests come in, the operating system also logs them into NVRAM. NVRAM is logically divided into two halves so that as one half is emptying out, the incoming requests are filling up the other half. They are going back and forth on that system. As soon as WAFL fills up one half of NVRAM, then WAFL forces a consistency point, or CP, to happen and it writes the contents of that half of NVRAM to the storage media. A fully loaded system will be doing back-to-back CPs so it is filling and refilling both halves of the NVRAM. Upon receipt from the host, WAFL logs writes in NVRAM and immediately sends an ACK (acknowledgment) back to the host. At that point from the host’s perspective, the data has been written to storage. But in fact, the data may be temporarily held in NVRAM. The goal of WAFL is to write data in full stripes across the storage media. To do this, it holds write requests in NVRAM while it chooses the best location for the data, does RAID calculations, parity calculations, and gathers enough data to write a full stripe across the entire RAID group.

8 NVRAM Operation (Cont.)
Client İşlemler Ana Hafıza Üzerinde Storage System Organize edilmiş data disklere blok halinde yazılır NVRAM sıfırlanır Main Memory NIC NIC N V R A M Main Memory + BATT - NVRAM operation nvram operation (cont.) WAFL will never hold data any longer than ten seconds before it establishes a CP. At least every 10 seconds, WAFL takes the contents of NVRAM and commits it to disk. As soon as a write request has been committed to a block on disk, WAFL clears it from the journal. On a system that is lightly loaded, an administrator can actually see the 10 second CPs happen; every ten seconds the lights cascade across the system. Most systems are running with a heavier load than that, and the CPs are happening every second, every four seconds, or every two seconds, depending on the system load. A question that frequently arises is, “is NVRAM a performance bottleneck?” No, it is not. The response time of RAM and NVRAM is measured in microseconds. Disk response times are always in milliseconds and it takes a few milliseconds for a disk to respond to an I/O. Disks are always the performance bottleneck of any storage system because disks are radically slower than any other component on the system such as CPU or RAM. When a system has gotten to the point that it is committing back-to-back CPs, the reason is because the disks are taking writes as fast as they possibly can. That is a platform limit for that system. If that platform limit is hit, then the option is to spread the traffic across more heads or upgrade the head to a system with greater capacity. That is a disk limitation; the disks are emptying NVRAM as quickly as possible. NVRAM could function faster if the disks could keep up.

9 NVRAM Sistem kapanmasında batarya ömrü “Planlı” Plansız” Haftarca
3-7 Gün Şarj Durumuna Göre Blinking LED Memory NVRAM5 Clustered Failover Ports Tavor 4x (10 Gb/s) InfiniBand 5.1Ah Battery (Dirty shutdown life –3-7 days) NVRAM benefits for both SAN and NAS environments nvram benefits for Both SAN and NAS Environments People often ask about battery life. If one does a dirty shutdown, that is, power off a running storage system without first halting it, the battery should last three to seven days. If the battery is partially discharged (less than 3.8 to 3.9 volts) such that it would not be able to keep NVRAM alive for three days, then Data ONTAP refuses to boot. It will charge the battery, and wait for it to get to a certain threshold before continuing. If one does a clean shutdown, the battery will last weeks or longer. The time it takes to charge the battery depends on how discharged it was. If it was fully discharged (< 3V), it takes approximately 10 hours to charge. If it is around the 3.8V threshold, that is, just low enough to keep the storage system from booting, it should be charged in about three to five hours, and Data ONTAP may allow the system to boot before then. Because its availability is critical to the uptime of the host controller, people often ask about Mean Time Between Failures (MTBF). It is approximately two million hours. NVRAM6 is a bit different in that two versions are offered. The versions differ in memory size as well as battery count. The card uses a total of three or five battery cells, depending on whether it is 512 MB or 2 GB. Note that the state of each of these batteries is independently known and may be queried by way of sysconfig –a. NVRAM6 512-MB/2-GB DIMM 3-Cell Battery IB CFO Connectors 2-Cell Battery (Added into 2GB version only) Feature NVRAM6 NVRAM5 Memory 512MB/2GB DDR PC2700 512MB DDR PC1600 PCI PCI-Express x8 64b/133MHz PCI-X 1.0 Battery One or two Li-Ion (5.9/9.8Ah) One Li-Ion 5.1Ah Power 25W 20W

10 NetApp Confidential – Limited Use
RAID 4 Parite Güncelleme Performans Ölçeklenebilirlik Yazma Zinciri RAID Stripe Parite Sürücüsü NetApp Confidential – Limited Use

11 Performans & Süreklilik
3 1 2 D D D D D P DP 3 1 2 3 9 7 1 1 2 1 5 12 2 3 1 2 8 12 1 1 3 2 7 11 Built into all NetApp storage arrays is a functionality called RAID-DP or RAID 6. This functionality, for the price of one additional disk drive provides 4000x more reliability than a traditional RAID 5 environment. RAID-DP protects against double disk failure, those crackles, hisses and jumps during a disk rebuild. Effectively the customer can have 2 disk fail at the same time and still carry on accessing data with minimal performance overhead. NetApp RAID-DP RAID 5’e Oranla 4,000x kat fazla koruma RAID 1+0 Oranla 75% verimli kapasite minimal performance Düşüşü (~0 to 3%) Diğer RAID Yapıları RAID 5 1 den fazla disk koruması yapamaz RAID 1+0 çok pahalı (2x disk) RAID 6 is Çok Yavaş (33% performans düşüşü)

12 NetApp Confidential – Limited Use
NetApp Donanımları FAS-Series, SA-Series, V-Series NetApp Confidential – Limited Use

13 NetApp Confidential – Limited Use
İş Sürekliliği “Active-Active Konfigurasyon Aynı İki Kontrol Ünitesi 8.0 Cluster-Mode ile 2’den Fazla Kontrol Üniteleri birbirinden Bağımsız Yönetilebilir NVRAM İçeriği InfiniBand ile birbirine Mirror Tüm Diskler Her İki Kontrol Ünitesine Bağlı Hata anında çalışan kontrol ünitesi diğerinin işlerini anında devralır. NetApp Confidential – Limited Use

14 Current Disk Layer technologies, part 1
FC Disk Üniteleri DS14 – 3U14D DS14mk4 FC FC diskler – ESH4 4Gbps DS14mk2 AT SATA diskler – AT-FCX 2Gbps Loop başına 6 Ünite (84 disks) FC/SATA farklı looplarda olmak zorunda NetApp Confidential – Limited Use

15 Current Disk Layer technologies, part 2
SAS Üniteler DS4243 – 4U24D, LFF, 3Gbps SAS SAS disk SATA disk SSD DS2246 – 2U24D, SFF, 6Gbps SAS Stack Başına 10 ünite (240 disk) SAS/SATA karışık stack olabilir SSD kendi stack yapısı içinde olmalı NetApp Confidential – Limited Use

16 NetApp Confidential – Limited Use
NetApp Yazılımları NetApp Confidential – Limited Use

17 NetApp Confidential – Limited Use
NetApp Yazılım Ailesi Temel yazılımlar Data ONTAP Aggregate ve FlexVol Snapshot SnapRestore FlexClone Deduplication Ek Yazılımlar SnapDrive SnapManager System Manager Operations Manager ve plug-in’leri NetApp Confidential – Limited Use

18 NetApp Confidential – Limited Use
Data ONTAP Open Networking Technology for Appliance Products Data ONTAP® NetApp Confidential – Limited Use

19 NetApp Confidential – Limited Use
Protokoller Tüm Platformlarda Data ONTAP Tüm Platformlar; FCP ve FCoE iSCSI CIFS NFS HTTP/S FTP, TFTP, SFTP NetApp Confidential – Limited Use

20 AutoSupport Olayları ASUP Raporlama ve bilgilendirme altyapısı:
Haftalık Loglar Sistemin olası problemlerine karşı proaktiv alarmlar ve önlemler Aşağıdakiler gerçekleşirse sistem yeniden başlar NVRAM batarya şarjı düşük veya boş Disk, fan, ve power supply hataları Disk Ünitesi Hataları Sistemin aşırı ısınması Cluster Hataları File system Büyümesi gereğinden fazla ise SNMP Hataları To improve security, NetApp supports Transport Layer Security (TLS) via VeriSign 256-bit digital certificates for encryption and authentication between mail server gateways. TLS encrypts the AutoSupport content between your server and the NetApp servers. NetApp has also added VeriSign digital certificates to all our servers to provide strong authentication against "man-in-the-middle" attacks. Please have your administrators enable these features on your server. Many servers support TLS and certificate-based authentication, so it is just a matter of enabling these features on your servers. NetApp’s automated system processes all AutoSupport messages received. If an error is detected from a system not covered under a support agreement, and we have the current contact information, an automatic reply will be sent to the customer requesting that support coverage be renewed on the system. NetApp will not provide any RMA (return material authorization) or technical assistance unless the system has a valid support agreement.

21 Data ONTAP Storage Katmanı: Aggregate
Aggregate — Bir ve Birden fazla Flexvol alanının çalışabilmesi için gerekli olan disk havuzudur Aggregate Fiziksel bir katmandır Aggregate RG 0 RG 1 Aggregate RG 2 Data ONTAP 7.0 storage terminology: aggregate What is an aggregate? An aggregate is a collection of disks. It can be multiple RAID groups or just one RAID group. It is a collection of physical disk space used as a container to support one or more volumes. An aggregate is the physical layer. When you create an aggregate, you do not have any usable space yet. There is still nothing for a host to connect to in an aggregate. Volumes must be created, either traditional or flexible, using the aggregate.

22 Data ONTAP 7.0 Storage Terminolojisi: Flexible Volume
Flexible volume — Bir Aggregate içerisinde oluşturulan esnek veri depolama alanlarıdır Flexible volume aggregate bağımlı değildir Flexible volume mantıksal bir katmandır. Aggregate FlexVol2 RG 0 FlexVol1 FlexVol2 RG 1 FlexVol1 Data ONTAP 7.0 Storage Terminology: Flexible Volume FlexVol2 RG 2 Data ontap storage terminology: flexible volume A Flexible volume is a collection of disk space allocated from the available space within the aggregate. FlexVol volumes are loosely tied to their aggregate and will be even more loosely tied in the future with the implementation of Data ONTAP GX functionality. Notice as the picture is showing, that both FlexVol volumes are striped across all of the disks of the aggregate. That is always true of a FlexVol volume no matter what the size. A FlexVol volume can be as small as 20 MB or as large as the entire aggregate, up to 16 terabytes in a current release of Data ONTAP. FlexVol1

23 Aggregate: 32-bit ve 64-bit
32-bit Aggregate 16TB maximum kapasitededir 64-bit Aggregate teorik olarak 64ZB (Zettabyte = 1m PB) adresleyebilir Gerçekte sistemin ram, işlemci yapısına göre sınırlandırılır Model Max 64-bit Aggr in 8.0 FAS6080 100 TB FAS6040 70 TB FAS3170 FAS3160 50 TB FAS3140 40 TB FAS2040 30 TB NetApp Confidential – Limited Use

24 FlexVols™: I/O Performansını Arttırır
Geleneksel Volume Yapısı: Volume performansı altındaki disk ve kafa sayısına bağlıdır Volume büyüdükçe disk hataları oluşur FlexVol: Aggregate içerisindeki tüm diskler kullanılır Flexvol boyutu önemli değildir. At the same time as increasing utilisation, customers will experience an increase in performance. Lets see how that works. When in a DAS each server has its own physical disks for the data, these disks spin at anywhere between 7,500 rpm and 15,000 rpm, most usually customers buy 10,000 rpm disks, so lets compare these. If we assume that a 10,000 rpm disk is capable of supplying the application 10Mb per second and we have 4 disks in the DAS based server. We can then assume we will get 40 Mb/s (excluding RAID overhead), to increase the performance more disks would need to be added, which could go against the physical storage requirements or the customer will face downtime while the data is transferred to quicker disks. With Aggregates and FlexVols, as we’ve seen, the data gets written across every disk in the aggregate regardless of the size of the FlexVol. So assuming we have 14 (one disk shelf), 7,500 rpm SATA disks, that can provide data to the server at a rate of 7.5 Mb/s we can see that the performance will increase. 7.5 Mb/s time 14 disks = 105 MB/s. Additionally if we wish to increase speed or grow the physical storage we can non-disruptively add disks to an aggregate and then the FlexVol automatically writes to the additional disks in the aggregate, therefore increasing performance and physical storage.

25 NetApp Confidential – Limited Use
Snapshot Teknolojisi Bir Çok yazılımın altyapısını oluşturur Dosya sisteminin anlık imajı’dır Volume tabanlıdır* Zamanlanmış, manuel veya uygulama entegre Sıfır performans kaybı Volume başına 255 adet ÜCRETSİZ Tüm DataONTAP sistemlerinde bulunur. *Metro Cluster yapılarında Aggregate tabanlı da çalışmaktadır. NetApp Confidential – Limited Use

26 Snapshot copy Kullanımı
Snapshot kopyaları okunabilir. Read-only yapıdadır Security metadata’sı korunur Snapshot kopyasından tek dosya çıkartılabilir NAS ortamlarında kullanıma hazırdır SAN ortamlarında geri dönüş için admin yetkisi gereklidir Snaprestore ile tüm snapshotlar geri dönülebilir. NetApp Confidential – Limited Use

27 NetApp Confidential – Limited Use
SnapRestore Bir volume’un tamamını bir snapshot noktasına döndürebilir Pointer işlemleri ile çalışır Anında dönüş sağlar Ekstra yer kaplamaz Sonraki snapshot kopyaları silinir Zamanda geri giderseniz gittiğiniz zamandan sonraki işlemler kaybolur Genelde Full volume restore için kullanılır ama tek dosya da dönülebilir İnode sayısına göre geri dönme işlemi zaman alabilir. NetApp Confidential – Limited Use

28 NetApp Confidential – Limited Use
FlexClone Yazılabilir bir Snapshot kopyasıdır Snapshot kopyasının bir kopyasını çıkartıp writable yapar Bir snapshot kopyasından 255 clone çıkabilir Oluşturulduğu snapshot kopyası clon silinene kadar silinemez Sadece yazılacak farklar için delta alan yer kaplar Clone snapshot kopyasından kopartılıp ayrı bir volume olabilir %100 yer kaplar NetApp Confidential – Limited Use

29 FlexClone – Nerelerde Kullanılır
Veritabanı Test/Dev kopyaları yer kaplamadan oluşturulur Bu kopyalar anında kullanıma hazırdır zaman kazandırır Sanallaştırma Birbirine benzer sanal makinalar anında ve yer kaplamadan oluşturulur Bir snapshot kopyasına writable erişilmek istenen her durumda kullanılır. NetApp Confidential – Limited Use

30 NetApp Confidential – Limited Use
Tekilleştirme Blok-Tabanlıdır. (4K) Bir FlexVol içerisindeki tüm blokları kapsar Aggregate tabanlı değildir Sadece volume içerisinde çalışır Protokol bağımsız çalışır Basitçe durdurulup kapatılabilir/açılabilir Performansa etkisi çok azdır. NetApp Confidential – Limited Use

31 Tekilleştirme Limitleri (TB)*
FAS Max Normal Max w/ dedupe Max data size 2020 16 1 17 2040 3 19 2050 2 18 3140 4 20 3160 32 3170 6040 6080 7.3.4 ve Üzeri / mode. OS için geçerlidir. NetApp Confidential – Limited Use

32 NetApp Confidential – Limited Use
Sıkıştırma Sadece versiyonunda Yeni Lisans (Ücretsiz) Deduplication etkin ama çalışmayacak şekilde İşlemci bağımlı, çalışma performans etkisi Tekilleştirme ile aynı limitlerde Ortama göre tekilleştirme ile birlikte çalşabilir. NetApp Confidential – Limited Use

33 MetroCluster – 100 km Kadar Uzayabilir
Şehirler Veya Bölgeler Arası MetroCluster Tek bir datacenter’ın birden fazla bölgeye bölünebilmesi FAS3000, FAS3100, FAS6000 V-Series Katlar, Binalar arası veya Kampüs Data Center İçinde MetroCluster – 100 km Kadar Uzayabilir MetroCluster can address your continuous availability requirements, whether it is deployed inside a data center; (click for animation) at different locations in a building; (click for animation) or across city or metro-wide deployments up to a maximum distance of 100km. This enables a level of availability that goes beyond the high-availability features in a single array which makes MetroCluster a highly versatile solution. MetroCluster supports NetApp FAS3000, FAS3100, FAS6000 and V-Series. HA Pair MetroCluster 500 m Kadar NetApp Confidential - Limited Use

34 Tek Datacenter için MetroCluster
Node 1 Node 2 Stretch MetroCluster 500m Cluster heartbeat is through InfiniBand Uses the NVRAM cards EXCEPT X1926A card for 31xx series <= 500 m when using point-to-point connection* Fibre Channel cabling Patch panels reduce the distance Disk shelves Supports up to platform limit 2Gb or 4Gb ATA supported Disk ownership same as Fabric MetroCluster *Must use OM3 type cabling (or better) to achieve 500 m Volume X Volume Y Volume Y Mirror Volume X Mirror Data Center NetApp Confidential – Limited Use

35 Şehir/Bölgeler Arası MetroCluster
Binası B Binası Fabric MetroCluster Volume X Volume Y Mirror <=100 km Volume Y Volume X Mirror DWDM Dark Fiber SPOF Yok– Tüm Parçalar ve Bağlantılar Yedekli Dedike Çoklu FC switchler Cluster Interconnect is VI over FC (versus SCSI) X1922A card for 3020, 3050 X1926A card for 3040, 3070, 31xx, and 6xxx Fibre Channel Switches Disk and controller interconnect Currently using Brocade 300 and 5100, Licensed for full (multiswitch) fabric Customer-supplied switches not supported Configuring for Long Distances Up to 10 km - 4 “Long wave” SFPs (X1670A-R5) Greater than 10 km - 4 “Extended long wave” SFPs; Brocade certified Extended distance license required in order to set Buffer credits set accordingly Storage ports can be 2Gbps - 2 dual-port FC HBAs OR 4 onboard ports 4Gbps - 4 onboard ports (3070, 6000 series) Quad-port FC HBAs Disk shelves Shelves on each loop must be same speed 2 shelves per loop maximum ATA shelves not supported Depending on FAS model, ownership determined by software or hardware rules Are attached to same ports on both switches (hardware ownership) NetApp Confidential - Limited Use

36 NetApp Confidential – Limited Use
Teşekkürler NetApp Confidential – Limited Use