Thursday, July 17, 2014

Root Volume Dangerously Low on Space

So I wanted to walk through a customer's data migration plan in my home lab to brush up on some SnapMirror and FlexClone commands in clustered ONTAP. To do this I would need a functional cluster. Which to my knowledge, I had a several ONTAP simulators running clustered ONTAP, and a couple functional clusters. It's been a week or more since I logged into anything in the home lab and have been neglecting/neglected the usual administrative duties that go along with managing storage and virtual machines. So needless to say, before I could do what I really wanted to do, I needed  to clean up my mess, so to speak.

My single-node cluster was unavailable via System Manager, so I opened the node's console from VSphere and was welcomed with the following:


Notice the console message "The root volume (/mroot) on miamicl-01 is dangerously low on space; less than 10 MB remaining. To make space available, delete old snapshot copies, delete unneeded files, or expand the root volume capacity. After space is made available, reboot the controller. If needed contact technical support for assistance." And also the message where the controller calls home with the following: "Call home for ROOT VOLUME NOT WORKING PROPERLY: RECOVERY REQUIRED."

The first error message seems pretty self-explanatory. The second message sounds scary, but in this case its a quick fix. Free up some space in the root volume on miamicl-01 and we will be back in business in the "Miami office" which is critical. Luckily I have run into this before and even had people ask how to manage space in a node's root volume as it is not accessible from System Manager. As far as I know the only way to manage a nodes root volume is via the nodeshell CLI interface.

So let's login and free up some space, shall we? As soon as I login I get the same message as above about low space on root volume. From the prompt (which doesn't look like the normal cluster shell) I enter node shell via the following to drop to the nodeshell:

miamicl-01::> system node run local
Type 'exit' or 'Ctrl+D' to return to the CLI

Now I need to determine the name of the root volume so I use "vol status". In the output in the "options" column I find vol0 has the root attribute. 

I then use "snap list vol0" to identify all snapshots in volume vol0. And I use "snap delete vol0 "  to delete all snapshots, replacing "" with "nightly.x" or "hourly.x" as necessary until I have deleted all snapshots. Then I reboot the node, and it comes up clean, I login, and am returned to the normal and familiar clustershell.

Great! My single node cluster is back online and now I can begin preparations of creating SnapMirror destinations and other tasks I need to familiarize myself with. But, let's take it a step further so next week when I login to test/verify some action plans or commands we don't have to recover the root volume. Let's configure auto-deletion of snapshots and a target free space to prevent this moving forward. I can now log the output using Putty/SSH instead of the VSphere console. And it looks like this:

login as: admin
Using keyboard-interactive authentication.
Password:
miamicl::>
miamicl::> node run local
Type 'exit' or 'Ctrl-D' to return to the CLI
miamicl-01> snap autodelete vol0 on
snap autodelete: snap autodelete enabled
miamicl-01> snap autodelete vol0 target_free_space 35
snap autodelete: snap autodelete configuration options set
miamicl-01> snap autodelete vol0
snapshot autodelete settings for vol0:
state                           : on
commitment                      : try
trigger                         : volume
target_free_space               : 35%
delete_order                    : oldest_first
defer_delete                    : user_created
prefix                          : (not specified)
destroy_list                    : none
miamicl-01>

This should prevent the root volume from having less than 35% free space. Now I can create SnapMirror relationships from my 7-mode vsim to my cluster. That should be fun so grab a beer and stick around, though I'm running short of time and probably won't get around to documenting it until next week. 

Wednesday, July 2, 2014

iSCSI SVM to ESXi Host - Clustered ONTAP

Eventually, I want to spin up a bunch of small VMs (probably will use Ubuntu as an example for now) on one volume and enable deduplication, or A-SIS (Advanced Single Instance Storage) to do this on a very small amount of disk space. In my lab, I will provision an iSCSI LUN to my ESXi host to use as the datastore to house my VMs using the cluster of Vsims running Clustered ONTAP.

Later, I will dedup that volume allowing me to create twice as many VMs as one would think could be created in that amount of disk space.

First, I login to the cluster via System Manager and create my SVM for iSCSI. In the left pane of System Manager, expand "Storage Virtual Machines", select the cluster, and click "create" to open the SVM setup wizard. This should look very similar to previous posts. Enter a name for your SVM. In my lab, currently, I'm using FlexVol volumes. For data protocols, select "iSCSI". Leave the language as the default. Select a security style. Select an aggregate to house the SVM's root volume. If you've previously entered DNS configuration information, these boxes will already be filled out. Mine looks like this:


Click "Submit and continue".
You can then configure the LIFs for iSCSI. In this lab I only made one LIF per node.
You can then enter SVM administrator details.
At this point your new iSCSI SVM is created. However, you still need to provision a volume, an iSCSI LUN, and map it to an igroup containing your ESX host's IQN.
Here's my igroup with my one ESX host's IQN:


Here's my LUNs view from System Manager showing my LUN mapped to the "esx" igroup:
You should be able to go into vSphere Client and highlight our host in the left pane. Then click on the "configuration" tab and click "storage". Click "add storage". Select  "Disk/LUN" and click "Next". Highlight the NetApp LUN we just presented and click "Next". You should now have a new iSCSI datastore mounted by your ESX host.

In my lab I created a 16Gb LUN and presented it to my ESX host. I've created one Ubuntu VM and I am already down to 10Gb free space in this LUN. I will create another one or two of these Ubuntu VMs and then enable dedup on that iSCSI volume to see if we can save some space. 









Tuesday, June 24, 2014

Creating a NAS vserver/Storage Virtual Machine - Clustered ONTAP

So if you've been following along, we installed the ONTAP simulator on ESX 5.1, and then we added the cluster to System Manager. We left off after we created new aggregates. We are now ready to carve these aggregates up into volumes and start creating vservers (now known as Storage Virtual Machines or SVMs) to serve data to clients. This post assumes the following:
1.You have a working/healthy cluster that you can access via System Manager and SSH.
2. You have a fully functional Windows domain with no DNS or other network infrastructure problems.
3. Be able to login to the domain from a workstation as a domain admin AND a domain user (non domain admin)
4. A test user account to login with to verify CIFS functionality.
5. A *nix box that you can mount NFS exports with.



So let's start with a NAS SVM that will serve up CIFS shares and NFS exports. You can create an SVM via System Manager or the CLI. I'll demonstrate both in this post.

Step 1. Create the new NAS SVM through System Manager.
We'll login to the cluster via System Manager and in the left pane expand "Storage Virtual Machines".
Then select the cluster, and click create in the right pane.
This brings up the Storage Virtual Machine Setup wizard/dialog.
Enter a name for the new NAS vserver.
I'm using FlexVol volumes for now. We'll explore infinite volumes later.
Check the boxes next to the protocols you want on this SVM. In this case I'm choosing both NFS and CIFS. (Don't worry we'll do some iSCSI later).
I leave the Language as the default - C.UTF-8
Security Style, I'm leaving as NTFS in this example.
Provide a root aggregate. 
Enter DNS search domain and name servers. Mine looks like this:
Click "Submit & Continue".
So now we need to provide network configuration information for the data LIFS. We have the option of using one LIF for CIFS and NFS or we can uncheck the "Retain the CIFS data LIFs configuration for NFS clients." box to provide configuration information of a seperate LIF for NFS. In this example we will just use the same LIF. Though in production I'm sure many use-cases would dictate segmenting this traffic across different LIFs and physical ports.

We also need to provide CIFS server configuration details. Name the CIFS server, provide the FQDN of the domain you're joining, which OU you want the CIFS server to belong in and username/password of a user who can join computers to the domain. Mine looks like this:
Click "Submit & Continue"
You have the option to provide SVM administration configuration details. We'll explore this feature later. For now I'm going to hit "skip" in the bottom left.
You should now see a "New Storage Virtual Machine" (SVM) Summary" window showing your CIFS server was created successfully and NFS service has been started as well as other details:
Now that we have an SVM created, lets provision some CIFS shares and verify user access.

Step 2. Lets provision a volume for CIFS
In the left pane "Storage Virtual Machines" should already be expanded. Expand the cluster, then expand the new SVM. Then expand "Storage" and click on "volumes".
Click "Create"
The "Create Volume" dialog appears.
Provide a name (meaningful names are very helpful)
Choose an aggregate to host the volume.
Click "OK"
Leave "Storage Type" set to NAS (I don't think there's any other option since we didn't enable iSCSI protocol on this SVM).
Enter the desired size.
Enter the desired snapshot reserve.
Select "Thin Provisioned" if desired. In my labs I almost ALWAYS thin provision as it saves me a lot of space.
We'll go over Storage Efficiency and QOS at a later time.
Click "Create"

Step 3. Create shares.
In the left pane of System Manager "Storage Virtual Machines" should be expanded, as well as the nas SVM, and "storage"
Click "Namespace"
Notice the volume we created for CIFS is automatically mounted under "/"


Click on "Shares" in the left pane
Click "Create Share"
The "Create Share" dialog appears.
Click "Browse" to choose the volume we created in step 2. Or type the path in the "Folder To Share" field.
Provide a share name.
Provide a comment (optional)
If the share were hosting Hyper-V VHDs over SMB we would check the box to enable continuous availability for Hyper-v and/or SQL.
Click "Create"
Step 4. Set some permissions.
In this example we will make the share read-only to domain users, and read/write to administrators.
Under our NAS SVM in the left pane of System Manager, click "shares"
Highlight the CIFS share and click "edit"
The "Edit cifs Settings" dialog appears
Click the "Permissions" tab
You can see "Everyone" has full control. Technically they need this if the desire is for them to be able to write to any other shares below /cifs_vol1 which we created above. But in this example we are going to restrict this share so domain users can only read the share and only domain admins can read/write.
So uncheck "Full Control" under "Permissions"
Uncheck "Change"
Click the "Add.." Button so we can add admins and give them full control.
Type \Domain Admins where "" is your domain.
Check "Full Control"
Click "Save and Close"

Step 5. Verify access.
Let's verify Domain Administrators can access the CIFS share and have read/write access.
Login to a workstation that is a member of the domain as a member of the domain admins group.
Open Windows Explorer
In the address bar, browse to the share. In my example it is: \\nas1\cifs
Create a new folder.


Open the new folder and create a new text document.
Now login to a workstation that is a member of the domain as a generic domain user, who is NOT a member of the domain admins group.
Open Windows Explorer
In the address bar, browse to the share.
Note you can read the share and see the new folder.
Open the new folder.
And note we can see the new text document we created.
Try and delete the existing text document. Or try creating a new document.
You should see the following error from Windows:
We have successfully created a CIFS share and laid down some basic permission settings on it and verified the access is working as we intended. This basic demonstration of CIFS configuration is complete. 

Now let's try NFS. Bear with me, I'm a Windows guy by default and have only been playing with Linux for a few years, and in a very limited/noobish kind of way at that. I find that things I can do in Windows in a few minutes, may take hours (worst case, days) in Linux. But I'm getting better all the time.

Step 6. Create an NFS export on the NAS SVM
We already provisioned a volume in step 2, so let's rinse and repeat.
The only additional step is after creating the volume, select it.
Click "edit" 
Change the drop down for "Security style" to UNIX
Click "Save and Close"


Step 7. Mount the NFS export
Login to the linux box and attempt to mount the newly created volume.
I got this:
miami@miamioffice:/mnt$ sudo mount -t nfs nas1:nfs_vol1 /mnt/nfs
mount.nfs: access denied by server while mounting nas1:nfs_vol1
DOH!
When we create a volume it is automatically mounted in the namespace. as we demonstrated above. However, it inherits the namespace export policy. To resolve this, in System Manager expand "policies" and click on "export policies".
Highlight the default policy. 
Click "Add Rule"
The "Create Export Rule" dialog appears.
In the "Client Specification" field enter 0.0.0.0/0 NOTE: This means any client on any subnet gets access via the protocols we define below with the permissions defined further below.
Check the box for "CIFS"
Check the box for "NFS"
Mine looks like this:
Click "OK"
Now try mounting the export from the Linux box again:
miami@miamioffice:/mnt$ sudo mount -t nfs nas1:nfs_vol1 /mnt/nfs
miami@miamioffice:/mnt$

Success! Although, this export policy rule is set wide-open...in many environments this is probably not desirable, but in my lab it's fine. We'll demonstrate more restrictive policies later.
Now I'll blow this SVM away and create the same/similar configuration from the command line.

Step 8. Create same SVM from command line.
There are many shorter commands to complete the same thing which I will demonstrate later, but the easiest is the "vserver setup" command. 

miamicl::> vserver setup
Welcome to the Vserver Setup Wizard, which will lead you through
the steps to create a virtual storage server that serves data to clients.

You can enter the following commands at any time:
"help" or "?" if you want to have a question clarified,
"back" if you want to change your answers to previous questions, and
"exit" if you want to quit the Vserver Setup Wizard. Any changes
you made before typing "exit" will be applied.

You can restart the Vserver Setup Wizard by typing "vserver setup". To accept a default
or omit a question, do not enter a value.

Vserver Setup wizard creates and configures only data Vservers.
If you want to create a Vserver with Infinite Volume use the vserver create comm


Step 1. Create a Vserver.
You can type "back", "exit", or "help" at any question.

Enter the Vserver name: nas1
Choose the Vserver data protocols to be configured {nfs, cifs, fcp, iscsi, ndmp}
Choose the Vserver client services to be configured {ldap, nis, dns}: ldap,dns
Enter the Vserver's root volume aggregate [aggr1]:
Enter the Vserver language setting, or "help" to see all languages [C.UTF-8]:
Enter the Vserver root volume's security style {mixed, ntfs, unix} [unix]: ntfs
Vserver creation might take some time to finish....

Vserver nas1 with language set to C.UTF-8 created.  The permitted protocols are

Step 2: Create a data volume
You can type "back", "exit", or "help" at any question.

Do you want to create a data volume? {yes, no} [yes]:
Enter the volume name [vol1]: cifs
Enter the name of the aggregate to contain this volume [aggr1]:
Enter the volume size: 1g
Enter the volume junction path [/vol/cifs]:
It can take up to a minute to create a volume...


Volume cifs of size 1GB created on aggregate aggr1 successfully.
Do you want to create an additional data volume? {yes, no} [no]:


Step 3: Create a logical interface.
You can type "back", "exit", or "help" at any question.

Do you want to create a logical interface? {yes, no} [yes]:
Enter the LIF name [lif1]: nas1_data
Which protocols can use this interface {nfs, cifs, iscsi}: nfs,cifs
Enter the home node [miamicl-01]:
Enter the home port {e0a, e0b, e0c, e0d} [e0a]: e0c
Enter the IP address: 192.168.1.15
Enter the network mask: 255.255.255.0
Enter the default gateway IP address: 192.168.1.1

LIF nas1_data on node miamicl-01, on port e0c with IP address 192.168.1.15 was created.
Do you want to create an additional LIF now? {yes, no} [no]:


Step 4: Configure DNS (Domain Name Service).
You can type "back", "exit", or "help" at any question.

Do you want to configure DNS? {yes, no} [yes]:
Enter the comma separated DNS domain names: tuna.corp
Enter the comma separated DNS server IP addresses: 192.168.1.11

DNS for Vserver nas1 is configured.

Step 5: Configure LDAP (Lightweight Directory Access Protocol).
You can type "back", "exit", or "help" at any question.

Do you want to configure LDAP? {yes, no} [yes]:
Enter LDAP Server IP address: 192.168.1.11
Enter LDAP Server port number [389]:
Enter LDAP Server Minimum Bind Authentication Level {anonymous, simple, sasl} [anonymous]:
Enter Bind DN (User) [none]:
Enter Base DN []:

LDAP for Vserver nas1 is configured.
Warning: The aggr-list for Vserver "nas1" is empty. Once aggr-list is configured, -max-volumes will be enforced.

Step 6: Configure NFS.
You can type "back", "exit", or "help" at any question.

NFS configuration for Vserver nas1 created successfully.

Step 7: Configure CIFS.
You can type "back", "exit", or "help" at any question.

Do you want to configure CIFS? {yes, no} [yes]:
Enter the CIFS server name [NAS1-MIAMICL]: nas1
Enter the Active Directory domain name: tuna.corp

In order to create an Active Directory machine account for the CIFS server, you must supply the name and password of a Windows account with sufficient privileges to add computers to the
"CN=Computers" container within the "tuna.corp" domain.
Enter the user name: administrator
Enter the password:

CIFS server "NAS1" created and successfully joined the domain.
Do you want to share a data volume with CIFS clients? {yes, no} [yes]:
Enter the CIFS share name [cifs]:
Enter the CIFS share path [/vol/cifs]:
Select the initial level of access that the group "Everyone" has to the share {No_access, Read, Change, Full_Control} [No_access]:

The CIFS share "cifs" created successfully.
Default UNIX users and groups created successfully.

Vserver nas1, with protocol(s) nfs,cifs, and service(s) dns,ldap has been configured successfully.

miamicl::>

It's way faster to configure the nas1 server via the CLI. Granted there are a few more steps to make it exactly as the same as the one we made in System Manager the commands to do so are pretty easy if you remember these four words, "tab is your friend" in cluster mode.

Monday, June 23, 2014

Provisioning Aggregates

In the previous posts we installed our ONTAP simulator and applied some basic configuration settings. We also connected to the cluster via System Manager. From within System Manager we can login to the cluster, expand "storage" in the left pane and take a look at our disks.
It looks like the vsim comes with 2 "shelves" of 14 disks each. You'll also notice the first 3 disks are already assigned to an aggregate, aggr0, the node miamicl-01's root aggregate. This leaves 11 spares to create another aggregate with. Below the 11 spares you'll notice another "shelf" of disks which have a state of "present". These disks have not been assigned to the node yet. So let's assign them to miamicl-01. This cannot be done from System Manager and must be done via command line. We're just going to assign all of the unowned disks to miamicl-01. But you can specify individual disks, all unowned disks, or all disks on a specific adapter. See the below example:
miamicl::> storage disk assign -disk miamicl-01:v4.* -owner miamicl-01

miamicl::> storage disk show
                     Usable           Container
Disk                   Size Shelf Bay Type        Position   Aggregate Owner
---------------- ---------- ----- --- ----------- ---------- --------- --------
miamicl-01:v4.16     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.17     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.18     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.19     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.20     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.21     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.22     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.24     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.25     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.26     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.27     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.28     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.29     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v4.32     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.16     1020MB     -   - aggregate   dparity    aggr0     miamicl-01
miamicl-01:v5.17     1020MB     -   - aggregate   parity     aggr0     miamicl-01
miamicl-01:v5.18     1020MB     -   - aggregate   data       aggr0     miamicl-01
miamicl-01:v5.19     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.20     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.21     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.22     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.24     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.25     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.26     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.27     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.28     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.29     1020MB     -   - spare       present    -         miamicl-01
miamicl-01:v5.32     1020MB     -   - spare       present    -         miamicl-01
28 entries were displayed.


Now we have assigned all the previously unowned disks to node miamicl-01. And if we refresh the disk page in System Manager we see all disks are assigned and have been made spares.


 We are ready to carve up an aggregate. Now how you want to do this really depends on what you're planning to do. Do you want one large aggregate or multiple smaller ones? For my lab it doesn't really matter, it gets tore down and rebuilt every few months anyway. But for this post I will make two, one via System Manager and one via the command line. 

From System Manager expand "Storage" and click on "Aggregates". Then click on "create" 
This will bring up the "Create Aggregate Wizard". 
Click "Next".
Name your aggregate (Its nice to use meaningful names instead of the auto-generated name)
Specify RAID Type, RAID-DP is default and what we'll use in this post.
Click "next"
Click the button "Select Disks"


The wizard automatically determines the minimum number of hot spares (in this case 1 disk) and leaves the remaining spares in a group (in this case 24). We select the group. If we had other spares of another type there would be a second group in this list:

In this post I'm creating two 12 disk aggregates, so I will drop the number of capacity disks to use from 24 down to twelve, leaving us another 12 spares to make a second aggregate with via CLI. 
Then I hit "save and close" 
And now I can change the size of my RAID groups. 
In this example I have a 12 disk aggregate , I could put them all in one RAID group, but I'm going to split this one into two RAID groups. I'll try to remember to illustrate why for you later. So I change my RAID group size to 6 and hit save and close, then hit create.

Now let's do it from the command line.

miamicl::*> storage aggregate create -aggregate aggr2 -diskcount 12
[Job 16] Job succeeded: DONE. Warning: Creation of aggregate "aggr2" has been initiated.  12 disks need to be zeroed before they can be added to the aggregate.  The process has been initiated.  Once zeroing completes on these disks, all disks will be added at once.  Note that if the system reboots before the disk zeroing is complete, the aggregate will not exist.

miamicl::*> storage aggregate show
Aggregate     Size Available Used% State   #Vols  Nodes            RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
aggr0        900MB   43.43MB   95% online       1 miamicl-01       raid_dp,
                                                                   normal
aggr1_2rg   7.03GB    7.03GB    0% online       0 miamicl-01       raid_dp,
                                                                   normal
aggr2           0B        0B    0% creating     0 miamicl-01       raid_dp,
                                                                   initializing
3 entries were displayed.

miamicl::*>

And storage aggregate show now displays aggr0, aggr1_2rg (created from SysMgr), and aggr2 which we just made in the CLI and is still initializing. Time to grab a beer while disks zero and aggregate creation completes. 

OK, aggrs are created, and I wanted to show you why I made an aggr with 2 RAID groups. DATA ONTAP uses RAID-DP so there is really 2 parity disks in each RAID group. This allows the RAID group to suffer double disk failures and remain online. Not like we're too concerned with data protection or losing disks to parity in this post, but I think it's worth demonstrating.

So above when we created aggr1_2rg using System Manager we made it out of two, six-disk RAID groups. I unowned my one spare and disabled disk autoassign to help demonstrate this. Check out how many disks have to be failed to bring the aggregate offline:

Aggregate aggr1_2rg (online, raid_dp, degraded) (block checksums)
  Plex /aggr1_2rg/plex0 (online, normal, active)
    RAID group /aggr1_2rg/plex0/rg0 (double degraded, block checksums)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   v4.22   v4    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      parity    FAILED          N/A                        1020/ -
      data      v4.24   v4    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      data      v4.25   v4    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      data      v4.29   v4    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      data      FAILED          N/A                        1020/ -

    RAID group /aggr1_2rg/plex0/rg1 (double degraded, block checksums)

      RAID Disk Device  HA  SHELF BAY CHAN Pool Type  RPM  Used (MB/blks)    Phys (MB/blks)
      --------- ------  ------------- ---- ---- ---- ----- --------------    --------------
      dparity   FAILED          N/A                        1020/ -
      parity    v5.29   v5    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      data      FAILED          N/A                        1020/ -
      data      v5.32   v5    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      data      v5.27   v5    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448
      data      v4.32   v4    ?   ?   FC:B   -  FCAL 15000 1020/2089984      1027/2104448


So after failing out 4 disks in aggr1_2rg, the aggregate is still online. Aggr2 which only has one RAID group would only be able to survive half that many disk failures. Out of my 12 1GB disks aggr1_2rg, with it's two RAID groups, has 7GB of usable space. While aggr2 with its 12 1GB disks lumped into one RAID group provides an additional 1.7GB of usable space (8.7GB total). There is a pretty good document on the NetApp Support Site that does a way better job of explaining RAID-DP and RAID groups than I ever could. Check it out here.

Upcoming articles will go over provisioning and testing a vserver, or storage virtual machine, for CIFS.

Add Cluster to System Manager

In my last post we walked through installing the ONTAP simulator on ESX. If you're following along, you should have a single node running 8.2.1 Cluster Mode. This post we will go through adding our cluster to System Manager so that we can perform some management tasks through the GUI. While I prefer the command line, especially in 7-mode, there are some tasks in Cluster Mode I find System Manager almost indispensable for.

This post assumes the following:
1. You have a running cluster that you can access via SSH to the cluster management LIF.
2. You have installed OnCommand System Manager 3.1 on the system you're managing the storage cluster from.

Step 1. We will need to enable SNMP on the cluster to allow System Manager to discover the cluster. Beginning in Ontap 8.2 SNMP versions 1 and 2c are disabled by default.
You need to SSH to the cluster management LIF (could also use console) and use the command line to accomplish this. Once logged into the cluster we will use the "system snmp community" command to create a new SNMP community named "public" as illustrated below.

miamicl::> system snmp community add -community-name public -type ro
miamicl::> system snmp community show

miamicl

        ro  public

Step 2. Now that we have created the "public" SNMP community in the cluster we can discover and add it to System Manager. Open System Manager and near the top left corner there is an icon labeled "Discover". Click "Discover".
In the "Discover Storage Systems" dialog box, enter your cluster management IP address, and click discover. It will take a few seconds to discover the cluster, but it should show up. Highlight it and click "Add Selected Systems"

Step 3. Let's login and see what we have to play around with. Your System Manager window should now have your cluster showing. Highlight it and click "Login"


Login as admin and use the admin password you used during cluster setup, and click "Sign In" And you should see something like this:
Now as you can see there are no aggregates, volumes, vservers (SVMs) or any way for the cluster to serve any data. In the coming posts we will work on that and begin to explore just what we can do with some of the features of Data ONTAP.





Installing Data ONTAP Simulator on ESXi 5.1.0

One of the best ways to get your feet wet with Data ONTAP is to get your hands on the Simulate ONTAP 8 software which allows you to install a "vsim", or virtual machine that runs Data ONTAP 8. It provides a great way to experiment with and learn about many of the features of Data ONTAP. The simulator can be downloaded from the NetApp support site NetApp customers and select partners. This guide assumes you already have the following:

1. You have already downloaded the vsim_esx-cm.tgz package (I'll be using Clustered ONTAP) from the support site.
2. Installed and configured your ESX host and can access it via SSH as well as through the vSphere client.

The following steps will have your simulator up and running Clustered ONTAP pretty quick. Let me know if you have questions, if I left something out, or if this guide helped you.

Step 1. The first thing we need to do is copy the vsim package to the datastore we'll be using. I used the vSphere client to accomplish this, though there are other ways. Click on your host in the left pane of vSphere Client> Select the configuration tab> and select storage in the Hardware pane on the left.Your screeen should look like this:

Right click the datastore you want to house the simulator and click "browse data store". Then click the icon with a green up arrow on top of a hard drive.
Select "upload file" and browse to the location you downloaded the simulator to and select it to begin the upload.

Step 2. Once the file has uploaded we need to untar it so we can use it. 
SSH to your ESX host and cd to the datastore we uploaded the vsim.tgz file to. In my case it looked like this:

~ # cd vmfs/volumes/storage
/vmfs/volumes/53a81f39-7f7d23f5-1240-001a64ca2d40 #


From here you will run the following tar command:
# tar xvf vsim_esx-cm.tgz

It should look something like this at first:
/vmfs/volumes/53a81f39-7f7d23f5-1240-001a64ca2d40 # tar xvf vsim_esx-cm.tgz
vsim_esx-cm/
vsim_esx-cm/cfcard/
vsim_esx-cm/cfcard/env/
vsim_esx-cm/cfcard/env/env
vsim_esx-cm/nvram
vsim_esx-cm/DataONTAP.vmdk
vsim_esx-cm/DataONTAP-flat.vmdk

It will take sometime for the vmdk files to unpack. This is a good time to go grab a beer ;)

Step 3. Once complete you can refresh the datastore browser and see the unpacked folder. I usually rename it to something more meaningful, then drill down and rename the vmx file to something meaningful as well, but this is optional.

Now its showtime! Let's fire up our simulator and lay down some basic configuration of the cluster. 

Step 4. Within the "browse datastore" window drill down to the vsim's vmx file, right click it, and select "add to inventory"


Name the vsim and click "next"
Select a resource pool. and click "next" For me this is simple as I'm only using one host in my lab and therefore only have one resource pool. 
On the "ready to complete" screen click "finish"
Close the datastore browser window.
You should now see a new, powered off ONTAP simulator vm in your inventory in the left pane of vSphere.
Right-click the new vm, mouse-over "power" then click "power-on".

At this point if all goes well, your ONTAP simulator vm should power on. In which case you can skip this next section. My vm however, would not power on. I received the following error:
Lucky for me I have run into this before the last time I rebuilt my lab. ESXi 5.1 no longer loads the vmkernel multiextent module by default. To load it I ran the following command on the ESX host via SSH:
vmkload_mod multiextent

You should see something like this:
~ # vmkload_mod multiextent
Module multiextent loaded successfully
And now you can can power on the vm.

Step 5. To apply some basic configuration, right-click the vm and select "open console"
We want to interrupt the boot process by hitting "ctrl+c" when prompted for the boot menu.
At the boot menu, type "4" and enter.


You will likely see some console spam and within it the system will ask "Zero disks, reset config and install a new file system?"
Hit "y" and enter
It will ask you again, "Zero disks, reset config and install a new file system?:"
Hit "y" and enter

The simulator will reboot to wipe config/disks.
When it reboots youll see some more console spam and the first time I did this (long ago) I thought the simulator was panicking and dumping core. It's not, its just wiping the disks.
Let it do it's thing. When it boots, it will automatically fire off the cluster setup script. You'll want to type "create" and hit enter.
We're standing up a single node cluster in this post. I may do another one with multiple nodes later, but there's some limitations with the simulator when it comes to HA, so for now, we'll just use a single node.
Type "yes" and hit enter
There will be a delay after you hit enter. But just let it work.
You will then be asked to name your cluster. Type a name and hit enter
You'll then be asked to enter the base license key. Type the key in and hit enter.
You'll then have an opportunity to add additional license keys. Lets skip this for now and get SSH working so we don't have to use the console and can cut/paste the other license keys.
You'll be prompted to set and confirm the admin password. 
Now configure the management interface. This interface would be used to manage the entire cluster in a multi-node cluster. As soon as the management interface is configured, you can use SSH to connect to the cluster and complete configuration.

SSH to your management interface that you configured above. Log in as admin and the password you set earlier. If you managed to lose/forget the password, go back to step 5.

Once you're logged in via SSH, just re-run the cluster setup script and hitting enter to keep the same settings you already applied, until you get to adding licenses, DNS setup, etc. After completing the setup script, your cluster is ready to serve some data. I'll have another post up soon about actually doing some evaluation of the many features of Data ONTAP.