hit counter script
IBM Elastic Storage System 3200 Hardware Planning And Installation Manual
  1. Manuals
  2. Brands
  3. IBM Manuals
  4. Storage
  5. Elastic Storage System 3200
  6. Hardware planning and installation manual

IBM Elastic Storage System 3200 Hardware Planning And Installation Manual

Hide thumbs Also See for Elastic Storage System 3200:
Table of Contents

Advertisement

Quick Links

Download this manual
IBM Elastic Storage System 3200
6.1.5
Hardware Planning and Installation
Guide
IBM
SC31-5733-00

Advertisement

Table of Contents
loading

Related Manuals for IBM Elastic Storage System 3200

Summary of Contents for IBM Elastic Storage System 3200

  • Page 1 IBM Elastic Storage System 3200 6.1.5 Hardware Planning and Installation Guide SC31-5733-00...
  • Page 2 IBM welcomes your comments; see the topic “How to submit your comments” on page xvii. When you send information to IBM, you grant IBM a nonexclusive right to use or distribute the information in any way it believes appropriate without incurring any obligation to you.

  • Page 3: Table Of Contents

    Planning for site preparation........................25 Site preparation and physical planning....................25 Planning for hardware installation......................25 Planning for racks..........................25 Reviewing IBM Elastic Storage System 3200 location guidelines............26 IBM Elastic Storage System 3200 model.................... 26 Planning for power for server canister....................27 Physical installation planning........................29...
  • Page 4 Installing the EMS server in a rack.......................47 Unpacking IBM Elastic Storage System 3200 (IBM SSR task)............47 Installing support rails for the IBM Elastic Storage System 3200 system (IBM SSR task)....48 Installing enclosures (IBM SSR task)....................53 Installing cable management assembly (IBM SSR task)..............56 Connecting the components (IBM SSR task)..................58...
  • Page 5 Appendix B. Switch VLAN configuration instructions.......... 107 Appendix C. Dual 24 port (48 ports) MGMT switch ESS configuration....115 Appendix D. Cables/Adapters for SSR activities..........119 Accessibility features for the system..............123 Accessibility features..........................123 Keyboard navigation..........................123 IBM and accessibility..........................123 Glossary......................125 Index........................ 133 Index........................ 133   v...
  • Page 6 vi  ...

  • Page 7: Figures

    4. Network topology............................19 5. IBM Elastic Storage System 3200 high-level architecture and topology..........20 6. High-level view of system management.....................21 7. Front view of IBM Elastic Storage System 3200 with drives installed............22 8. Power module..............................22 9. SSR access port............................23 10.
  • Page 8 42. Installing outer CMA arm connector on outer rail connector..............57 43. Installing CMA body connector (E) on the left CMA body rail..............58 44. Orientation of ports on IBM Elastic Storage System 3200..............59 45. SSR access port / mini USB........................60 46.
  • Page 9 68. Configuring VLAN tag..........................74 69. Confirming VLAN tag..........................74 70. SSR Checks for p9 EMS..........................74 71. P9 EMS checks menu..........................77 72. P9 EMS SSR menu.............................78 73. System status LEDs on an IBM Elastic Storage System 3200 enclosure..........83   ix...
  • Page 10 74. Displaying system status LEDs on an IBM Elastic Storage System 3200 enclosure......83 75. Drive status LEDs on an IBM Elastic Storage System 3200 drive carrier..........85 76. IBM Elastic Storage System 3200 fan LEDs.....................86 77. IBM Elastic Storage System 3200 power module status LED..............87 78.
  • Page 11 99. Adapter slot to access server via Ethernet.................... 121   xi...
  • Page 12 xii  ...

  • Page 13: Tables

    7. Power specifications per power module....................31 8. Temperature requirements......................... 32 9. Physical characteristics of the enclosures....................32 10. Rack space requirements for the IBM Elastic Storage System 3200............33 11. Service clearance requirements....................... 35 12. Model 5141-FN1 measurements......................35 13. Clearances..............................35 14.
  • Page 14 24. IBM Elastic Storage System 3200 fan LEDs.....................86 25. IBM Elastic Storage System 3200 power module status LED..............87 26. IBM Elastic Storage System 3200 Ethernet port LEDs................89 27. Behavior of IBM Elastic Storage System 3200 server canister status LEDs .......... 89 xiv  ...

  • Page 15: About This Information

    This information is intended for administrators of IBM Elastic Storage System (ESS) that includes IBM Spectrum Scale RAID. IBM Elastic Storage System information units IBM Elastic Storage System (ESS) 3200 documentation consists of the following information units. Information unit Type of information Intended users...

  • Page 16: Conventions Used In This Information

    • IBM POWER9 servers, see https://www.ibm.com/docs/en/ess/6.1.2_ent?topic=guide-5105-22e- reference-information. For the latest support information about IBM Spectrum Scale RAID, see the IBM Spectrum Scale RAID FAQ in IBM Documentation. Conventions used in this information Table 1 on page xvi describes the typographic conventions used in this information. UNIX file name conventions are used throughout this information.

  • Page 17: How To Submit Your Comments

    In the left margin of the document, vertical lines indicate technical changes to the information. How to submit your comments To contact the IBM Spectrum Scale development organization, send your comments to the following email address: scale@us.ibm.com About this information  xvii...
  • Page 18 Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 19: Chapter 1. Notices

    This information was developed for products and services offered in the US. This material might be available from IBM in other languages. However, you may be required to own a copy of the product or product version in that language in order to access it.

  • Page 20: Trademarks

    International Business Machines Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the web at Copyright and trademark information at www.ibm.com/legal/copytrade.shtml.

  • Page 21: Homologation Statement

    DANGER: A danger notice indicates the presence of a hazard that has the potential of causing death or serious personal injury. (D002) 2. Locate the IBM Systems Safety Notices with the user publications that were provided with your system hardware.
  • Page 22 Electrical voltage and current from power, telephone, and communication cables are hazardous. To avoid a shock hazard: If IBM supplied the power cord(s), connect power to this unit only with the IBM provided power cord. Do not use the IBM provided power cord for any other product. Do not open or service any power supply assembly.
  • Page 23 plate. For racks with a DC power distribution panel (PDP), connect the customer’s DC power source to the PDP. Ensure that the proper polarity is used when attaching the DC power and DC power return wiring. • Connect any equipment that will be attached to this product to properly wired outlets. •...
  • Page 24 The weight of this part or unit is more than 29.3 kg (64.1 lb). It takes two persons to safely lift this part or unit. CAUTION: To avoid personal injury, before lifting this unit, remove all appropriate subassemblies per instructions to reduce the system weight. (C012) 6  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 25 CAUTION: CAUTION regarding IBM provided VENDOR LIFT TOOL: • Operation of LIFT TOOL by authorized personnel only • LIFT TOOL intended for use to assist, lift, install, remove units (load) up into rack elevations. It is not to be used loaded transporting over major ramps nor as a replacement for such designated tools like pallet jacks, walkies, fork trucks and such related relocation practices.
  • Page 26 • Ensure that the four leveling pads are raised to their highest position. • Ensure that there is no stabilizer bracket installed on the rack cabinet during movement. • Do not use a ramp inclined at more than 10 degrees. 8  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 27 • When the rack cabinet is in the new location, complete the following steps: – Lower the four leveling pads. – Install stabilizer brackets on the rack cabinet or in an earthquake environment bolt the rack to the floor. – If you removed any devices from the rack cabinet, repopulate the rack cabinet from the lowest position to the highest position.

  • Page 28: Special Caution And Safety Notices

    General safety When you service the IBM Elastic Storage System 3200 , follow general safety guidelines. Use the following general rules to ensure safety to yourself and others. • Observe good housekeeping in the area where the devices are kept during and after maintenance.
  • Page 29 Important: Good judgment must also be used to identify potential safety hazards due to the attachment of non-IBM features or options that are not covered by this inspection guide. If any unsafe conditions are present, you must determine how serious the apparent hazard might be and whether you can continue without first correcting the problem.

  • Page 30: Environmental Notices

    Attention: Depending on local conditions, the sound pressure can exceed 85 dB(A) during service operations. In such cases, wear appropriate hearing protection. Environmental notices This information contains all the required environmental notices for IBM Systems products in English and other languages. The IBM Systems Environmental Notices includes statements on limitations, product information, product recycling and disposal, battery information, flat panel display, refrigeration and water-cooling systems, external power supplies, and safety data sheets.

  • Page 31: Canada Notice

    IBM Elastic Storage System 3200 meets all regulatory EMC compliance requirements as offered, including auxiliary equipment and cables that may be ordered from IBM to be used in conjunction with IBM Elastic Storage System 3200. Canada Notice CAN ICES-3 (A)/NMB-3(A)

  • Page 32: Japan Electronics And Information Technology Industries Association (Jeita) Notice

    This statement applies to products greater than 20 A, single phase. This statement applies to products greater than 20 A per phase, three-phase. Japan Voluntary Control Council for Interference (VCCI) Notice 14  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 33: Korea Notice

    Korea Notice People's Republic of China Notice Russia Notice Taiwan Notice IBM Taiwan Contact Information: Chapter 1. Notices  15...

  • Page 34: United States Federal Communications Commission (Fcc) Notice

    Properly shielded, grounded, and approved cables and connectors must be used in order to meet FCC emission limits. IBM is not responsible for any radio or television interference caused by using other than approved cables and connectors, or by unauthorized changes or modifications to this equipment.

  • Page 35: Chapter 2. Technical Overview

    SSD drives to provide significant performance improvements as compared to SAS-attached flash drives. IBM Elastic Storage System 3200 can contain up to 24 NVMe-attached SSD drives. You can either use 12 drives in a half-populated configuration or 24 drives in a fully populated configuration. These drives are accessible from the front of IBM Elastic Storage System 3200, as shown in the following figure.
  • Page 36 Class Support The major characteristics of the IBM Elastic Storage System 3200 system are as follows: • IBM Spectrum Scale software with enclosure-based, all-inclusive software feature licensing. • NVMe transport protocol for high performance of 2.5-inch (SFF) NVMe-attached SSD drives: –...

  • Page 37: Ibm Elastic Storage System 3200 Networking

    Front View RHEL 8.2 Figure 3. IBM Elastic Storage System 3200 Networking The IBM Elastic Storage System 3200 has two adapters per server canister that consists of the following features: • AJZL: CX-6 InfiniBand/VPI in PCIe form factor – InfiniBand - HDR200 200 Gb / HDR100 100 Gb / EDR 100 Gb –...

  • Page 38: Ibm Elastic Storage System 3200 High-Level Architecture And Topology

    • The NVMe transport protocol supports industry standard NVMe flash drives. Enterprise-level software and support IBM Elastic Storage System 3200 consists of two I/O server canisters that run the IBM Spectrum Scale software, which is a part of the IBM Spectrum Storage family. For more information about the IBM Spectrum Scale capabilities, see IBM Spectrum Scale.

  • Page 39: Drives

    You need a minimum of one POWER9 EMS node as a part of your ESS cluster. The POWER9 EMS node can be ordered as a part of IBM Elastic Storage System 3200. When IBM Elastic Storage System 3200 is added to an existing ESS cluster that has a POWER9 EMS node, the same POWER9 EMS node can also be used to manage IBM Elastic Storage System 3200.

  • Page 40: Drive Slots

    • Set the IP address of the management interface. For more information, see “Best practices for network configuration (Customer reference task)” on page 61. Each of the canisters in the IBM Elastic Storage System 3200 has an USB mini-B SSR access port, as shown in the following figure.

  • Page 41: Hardware Specification Sheets

    Hardware specification sheets EMS node specifications IBM Elastic Storage System 3200 uses one EMS node in the basic configuration. The EMS node acts as the management server and provides infrastructure for hardware monitoring and hosts GUI. The EMS server has the following roles: •...
  • Page 42 24  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 43: Chapter 3. Planning For Hardware

    For detailed rack specification information on the 7965-S42 rack, see: Planning for the 7965-S42 rack. If you do not have a 7965-S42 rack and want to install the system into a non-IBM rack, see “ IBM Elastic Storage System 3200 physical environment requirements” on page 30. Ensure that the physical environment meet the specified requirements, such as rack space, power, and environmental conditions.

  • Page 44: Reviewing Ibm Elastic Storage System 3200 Location Guidelines

    IBM Elastic Storage System 3200 contains two I/O server canisters. Each IBM Elastic Storage System 3200 cluster consists of the following components: • One or more IBM Elastic Storage System 3200 systems – each system requires 2U (standard rack units) in a rack.

  • Page 45: Planning For Power For Server Canister

    Attention: The power cord is the main power disconnect. Ensure that the socket outlets are located near the equipment and are easily accessible. The following figure shows the back of a IBM Elastic Storage System 3200. Each power module is located on the sides of IBM Elastic Storage System 3200.
  • Page 46 Table 4. Power cord feature codes (continued) Feature codes Description END1 Power Cord 2.7 m (9 ft), Drawer to IBM PDU (250V/ 10A) C13/C14 (6671) 6672 Power Cord 2 m (6.5 ft), Drawer to IBM PDU (250V/ 10A) C13/C14 6665 Power Cord 2.8 m (9.2 ft), Drawer to IBM PDU...

  • Page 47: Physical Installation Planning

    Supported hosts IBM Spectrum Scale needs to be installed on the host server before it is connected to IBM Elastic Storage System 3200. IBM Spectrum Scale provides the high-performance scale-out clustering capabilities. For a list of supported host type and OS, see IBM Spectrum Scale Frequently Asked Questions.

  • Page 48: Ibm Elastic Storage System 3200 Physical Environment Requirements

    Attempting to move the drawer partially or completely out of the rack might cause the rack to become unstable or cause the drawer to fall out of the rack. (R001 part 2 of 2) Important: In addition, remember: 30  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 49 If there is less airflow, additional mechanisms are required to cool the enclosure. An appropriate IBM rack configuration would be the 7965-S42 IBM Rack Model S42, with standard rear door and feature code 6069 Front Door For 2.0 Meter Rack (High Perforation).
  • Page 50 The weight of this part or unit is more than 29.3 kg (64.1 lb). It takes two persons to safely lift this part or unit. The following table shows the rack space requirements for IBM Elastic Storage System 3200 in tabular form:...
  • Page 51 Learn about the requirements and specifications for 19-inch racks. These requirements and specifications are provided as an aid to help you understand the requirements to install IBM systems into racks. It is your responsibility, working with your rack manufacturer, to ensure that the rack that is chosen meets the requirements and specifications that are listed here.
  • Page 52 Figure 13. Sample illustration to show space around the rack Figure 14. Illustrating enclosure service envelope (Dimensions in millimeters) 34  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 53 For IBM Elastic Storage System 3200, use the recommended measurements that are given in the following table. Table 11. Service clearance requirements Front Back 1200 mm (47 in.) 1000 mm (39 in.) Storage racks require larger service clearances in the front of the rack.
  • Page 54 The mounting rails and mounting hardware that are provided with IBM products are designed and tested to safely support the product during operation and service activities, and to safely support the weight of your drawer or device.

  • Page 55: Acoustical Declaration With Noise Hazard Notice

    Supported drives The following table provides drive specifications for your IBM Elastic Storage System 3200 system. Table 16. Drive specifications Model and type 2.5-inch drives IBM Elastic Storage System 3200 with 24 2.5-inch 24 dual port NVMe drives (3.84 TB, 7.68 TB, 15.36 TB) drive slots For more information about the feature codes of the NVMe drives, see Table 3 on page 18.
  • Page 56 The general safety requirements for IBM products that are installed in non-IBM racks are as follows: • Any product or component that plugs into either an IBM power distribution unit or mains power (by using a power cord), or uses any voltage over 42 V ac or 60 V dc (considered to be hazardous voltage) must be Safety Certified by a Nationally Recognized Test Laboratory (NRTL) for the country in which it is installed.
  • Page 57 Grounding method must comply with applicable country's electric code (such as NEC or CEC). Ground continuity can be verified by your IBM service personnel, after the installation is completed, and must be verified before the first service activity.

  • Page 58: Ip Address Allocation And Usage

    (such as in rack doors), it must be safety glass. If wood shelves are used in the rack/cabinet, they must be treated with a UL Listed flame-retardant coating. • The rack or cabinet configuration must comply with all IBM requirements for "safe to service" (contact your IBM Installation Planning Representative for assistance in determining if the environment is safe).

  • Page 59: Planning For 100 Gbps Adapter

    Planning for cables Connections for IBM Elastic Storage System 3200 Care must be taken to note the orientation of each server canister in IBM Elastic Storage System 3200 so that the interconnect cables are properly connected. IBM Elastic Storage System 3200 contains two server canisters. A label on IBM Elastic Storage System 3200 identifies each server canister and power module.

  • Page 60: Orientation Of Ports On Ibm Elastic Storage System 3200

    Mini Display Port Mini Display Port Figure 17. Orientation of ports on IBM Elastic Storage System 3200 The Ethernet port on each server canister provides system management and BMC connections. The onboard 1 Gbps Ethernet port use RJ-45 connection. A separate technician port provides access to each I/O server canister for hardware check and setup.

  • Page 61: Planning For Adapters

    Each IBM Elastic Storage System 3200 system requires two Ethernet cables to connect it to an Ethernet switch. One cable connects to the Ethernet port of the left server canister, and the other cable connects to Ethernet port of the right server canister.

  • Page 62: Supported Environment

    Supported environment Refer to the IBM support website for up-to-date information about the supported environment for your system. Environmental topics can include updates about the following items: • Host attachments • Switches • Firmware levels • Other support hardware 44  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 63: Chapter 4. Installing

    Chapter 4. Installing This section covers how to rack, stack, unpack, and install IBM Elastic Storage System 3200. It also guides the customer on how to fill out the installation worksheet. The service support representatives (SSR) can refer to “Completing the ESS 3200 installation at the customer site (SSR task)”...
  • Page 64 Work your way up the rack to additional ESS 3200 systems. If the EMS is already a part of an existing ESS setup and only IBM Elastic Storage System 3200 nodes are being added, then perform a ping test from each canister to the EMS. Ask the customer for the management IP of the EMS that you target.

  • Page 65: Detailed Installation Steps (Ssr Task)

    Unpacking IBM Elastic Storage System 3200 (IBM SSR task) Before you unpack IBM Elastic Storage System 3200, ensure that you have reviewed and followed all the related instructions. The IBM Elastic Storage System 3200 system and related parts are shipped preinstalled in the enclosure: •...

  • Page 66: Installing Support Rails For The Ibm Elastic Storage System 3200 System (Ibm Ssr Task)

    6. Remove the four corner reinforcement pieces from the carton. 7. Using the box knife, carefully cut the four corners of the carton from top to bottom. 8. Fold the sides and back of the carton down to uncover the rear of the IBM Elastic Storage System 3200 system.

  • Page 67: Rail Assembly

    Rear Inner Rail Outer Rail Front Figure 18. Rail assembly Note: The rail kit accommodates racks with square or round rack post holes. Preparation for each type is nearly identical to the exception that round post holes would require different set of screws. No changes are necessary for racks with square rack post holes.

  • Page 68: Release Tab

    Release the catch to secure the rail to the post once the screws are fully inserted. d) Align the left front bracket mounting screws with the appropriate rack post holes in the front left rack post. Ensure that the rail is level before you continue. 50  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 69 e) Secure the rear end of the left outer rail to the rack using the M5 shipping screw that comes with the supplier rail kit. Figure 24. Securing the rear end of the left outer rail f) Remove the middle screw (as shown in the following figure) from the front end of the left outer rail using a flat tip screwdriver.
  • Page 70 Figure 29. Standoffs secured over the keyholes c) Secure the rail in place by using one of the M4 screws in the hole just behind the release tab. See the following figure. 52  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 71: Installing Enclosures (Ibm Ssr Task)

    Following your enclosure location plan, install the IBM Elastic Storage System 3200 system. CAUTION: • To lift an IBM Elastic Storage System 3200 system requires at least two people. To reduce the enclosure weight to permit one person to safely lift the enclosure, you must temporarily remove the server canisters, power modules, and all drives from the enclosure.
  • Page 72 Install the two screws on the rear side of the left bezel ear cap as shown in the following figure. Figure 34. Displaying left ear cap front view c) Repeat the above steps to install the right bezel ear cap on the right side of the enclosure. 54  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 73 M5 X 9 shipping screws as shown in the following figure. Figure 37. Securing the enclosure At this point, the rails and IBM Elastic Storage System 3200 enclosure are installed. The enclosure is mounted on the rail with all the components that were removed, reinserted into the slots.

  • Page 74: Installing Cable Management Assembly (Ibm Ssr Task)

    2. To change the direction of the arm for use on the opposite side of the enclosure, press the release buttons on the outside of the CMA elbow and rotate the arm 180 degrees. 56  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 75 Figure 40. CMA release buttons 3. Slide the inner CMA arm connector (A) onto the lower right inner rail connector (B) as shown in the following figure. Figure 41. Installing inner CMA arm connector on inner rail connector 4. Slide the outer CMA arm connector (C) onto the lower right outer rail connector (D) as shown in the following figure.

  • Page 76: Connecting The Components (Ibm Ssr Task)

    Connecting the components (IBM SSR task) After installing rails and the IBM Elastic Storage System 3200 enclosure in the rack, the next step is to connect to the power and network. Connecting Ethernet cables to the server canisters To provide the system management connectivity for your system, you must connect the Ethernet cables to Ethernet port of each server canister in IBM Elastic Storage System 3200.

  • Page 77: Connecting A Mini Usb Cable To The Ssr Access Port (Ibm Ssr Task)

    Connecting a mini USB cable to the SSR access port (IBM SSR task) Before you check the system, you must connect a mini USB cable to the SSR access port on the IBM Elastic Storage System 3200. 1. Locate the SSR access ports on IBM Elastic Storage System 3200, as shown in the following figure:...
  • Page 78 4. If extra cable length is needed, use a mini USB female to male extension cable for canisters that are higher up in the frame. Figure 48. Mini USB female to male extension cable 60  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 79: Install And Connect Management Server 5105-22E

    • For information about 5147-SCx networking cabling, see ESS SCx network cabling on the IBM Worldwide Customized Installation Instructions (WCII) website. • For information about 5147-SLx networking cabling, see ESS SLx network cabling on the IBM Worldwide Customized Installation Instructions (WCII) website.

  • Page 80: Prerequisites For Installation Completion By Ssr

    SSR sets on each node. If the worksheet is not completed, the SSR might confirm with the personnel at the data center and use default IP addresses listed. Racked or rackless installation: 62  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 81 • If the system did not come racked, the SSR would rack the necessary components before the installation starts. • IBM Elastic Storage System 3200 needs to be placed in the rack location provided by the user. For example, EIA 1-2 and 3-4.

  • Page 82: Completing The Ess 3200 Installation At The Customer Site (Ssr Task)

    This document is intended to be used by the SSR at the customer site to complete installation. The document is for new customers of IBM Elastic Storage System 3200 only (EMS + one or more IBM Elastic Storage System 3200 units).

  • Page 83: Ssr Access Port/Mini Usb

    – FSP/BMC IP (C11-T2) – HMC1 interface Note: If there are more than one IBM Elastic Storage System 3200 being deployed, the customer needs to provide Canister management and BMC interface IP addresses for each additional unit (as well as potentially the VLAN tag).

  • Page 84: Putty Tool

    Figure 51. PuTTY tool d. Power on IBM Elastic Storage System 3200 canister 1 (left). You must press the power button to boot the system up (see Figure 17 on page 42). The system begins to boot and directs to a login prompt.

  • Page 85: The Essutils Tool

    e. Log in by using the following SSR credentials and press the Enter key. uid: essserv1 pw:<{Enclosure serial number}A|B> Add a ‘A’ at the end of the enclosure serial number for the left canister for password (use 'B' for right canister). For example, if the enclosure serial number is 78E00TF, the password for the left canister will be 78E00TFA.

  • Page 86: Ssr Tools

    We default the session to the following sizes: stty rows 40 cols 130 As a workaround you can log in as root (password = ibmesscluster) and do the following: 68  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 87 Adjust the stty (use a larger value for rows/cols). For example, stty rows 50 cols 150 Then press the Enter key. Now execute essutils as root to run through the code 20 flow. You can also edit the essserv1 profile with this value and log back in as essserv1. vi /home/essserv1/.profile Edit the value and save.

  • Page 88: Quick Storage Configuration Check

    Figure 60. Quick storage configuration check window The output displays with a PASS or ERROR message. If the step fails with an error message, debug the issue and try again. 70  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 89 Press the Enter key to continue. Note: There is a known issue with this check that might result in the following error. If you see the "Unsupported NVMe Drive.." message, it is likely that the code is incorrectly looking for a "G3" NVMe Drive type. To confirm this is not a real error do the following: i) Exit essutils and log in as the root user.

  • Page 90: Recovery Group Descriptor

    Highlight the Set FSP ipmi IP address (EMS or 3200) menu, press c key and then press the Enter key. Figure 65. Setting FSP ipmi IP address (EMS or 3200) 72  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 91: Setting Fsp Ipmi Static State And Netmask (Ems Or 3200)

    A new panel opens. This new panel allows the SSR to change the IP address. Press the Ctrl + G keys to save the changes. Press the Enter key to execute. This sets the BMC IP address. For information, see the installation worksheet.
  • Page 92 In most situations (Especially if brand new racked system and IBM Cumulus switch), you will press Enter immediately to use VLAN tag 101. Figure 68. Configuring VLAN tag r. Highlight the Confirm BMC VLAN tag menu and press Enter. This confirms the VLAN tag set in the previous option.
  • Page 93 IBM service. 7. Unplug the serial connection and insert to the canister 2 (right). 8. For IBM Elastic Storage System 3200 canister 2 (right), repeat the configuration steps of Canister 1 (left).

  • Page 94: Checking Ess Management Server And Protocol Nodes

    – HMC1 port – FSP VLAN – C11-T2 – FSP VLAN • Use P9 EMS panel in essutils for checking the p9 EMS. Do not use the ESS 3200 panel. 76  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 95 Perform the following steps if a new POWER9 EMS is included in the configuration: Note: Skip this section if the EMS has already been configured or if you are only adding additional IBM Elastic Storage System 3200 nodes to the existing setup.

  • Page 96: P9 Ems Ssr Menu

    Note: Fix passwordless ssh, Gather serviceable event ID details (EMS ONLY), and Erase serviceable event ID (EMS ONLY) are only used if needed. If the ping was successful to the C11-T4 port proceed to login by using the following credentials. User ID: esserv1 78  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 97 Password: <Serial number of the EMS (all caps)> Then press the Enter key twice to bring up essutils. Once logged in, select SSR Tools and then ESS 3200 SSR checks. b. Assign IP addresses. i) Set the following IP addresses of the EMS (On the protocol node you will only set two IP addresses –...
  • Page 98 If the ping is not successful, check the cabling and switch. You might need to log in as root to confirm whether the IP address was set correctly. For more information, see IBM Elastic Storage System 5000: Problem Determination Guide.

  • Page 99: Protocol Node Checks

    Protocol node checks The p9 protocol node requires very similar checks as the p9 EMS. The only difference is skip the following: • Assign IP address to FSP Interface (EMS ONLY) (C11-T2) • Check FSP interface (EMS ONLY) (C11-T2) Perform any ping tests from the EMS if applicable (thus protocol node setup would be done before EMS if a new environment).
  • Page 100 82  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 101: Chapter 5. Monitoring The System Using Leds

    The following figure shows system status LEDs on an IBM Elastic Storage System 3200 enclosure: Figure 73. System status LEDs on an IBM Elastic Storage System 3200 enclosure The following figure shows the location of each LED available on an IBM Elastic Storage System 3200 enclosure from left to right: Figure 74.

  • Page 102: Drive Carrier Assembly Leds

    Drive carrier assembly LEDs Each drive carrier assembly of IBM Elastic Storage System 3200 includes a set of LEDs that are visible from the bottom of the drive carrier. On system startup, the green LED illuminates automatically.

  • Page 103: Fan Module Leds

    The SSD does not have any power. Fan module LEDs The IBM Elastic Storage System 3200 includes three LEDs with each of the six fan modules. The available LEDs are of blue, amber, and green color. These LEDs are used to monitor fan modules.

  • Page 104: Power Module Leds

    A fan module fault is detected. Power module LEDs These LEDs are used to monitor power modules of the IBM Elastic Storage System 3200. The power module status LEDs are visible from the rear of the enclosure. The following figure provides a close view of the power module status LED.

  • Page 105: Server Canister Operator Panel Leds

    Server canister operator panel LEDs The IBM Elastic Storage System 3200 server canister consists three status LEDs on the bottom of the server canister operator panel. In addition, two self-driven Ethernet port LEDs are located to the right of the server canister status LEDs.

  • Page 106: Ethernet Port Leds

    Figure 78. IBM Elastic Storage System 3200 server canister operator panel The following figure shows a close view of server canister operator panel LEDs. Figure 79. Close view of server canister operator panel LEDs Ethernet port LEDs The GbE port on the rear of server canister is a standard RJ45-style connector. These are self-driven LEDs indicate speed and link activity as described in Table 26 on page 89.

  • Page 107: Server Canister Status Leds

    Figure 80. Close view of IBM Elastic Storage System 3200 Ethernet port LEDs The following table describes the behavior of Ethernet port LEDs. Table 26. IBM Elastic Storage System 3200 Ethernet port LEDs Left LED Right LED Behavior Indicates no link.
  • Page 108 90  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 109: Chapter 6. Troubleshooting (Ssr Tasks)

    Chapter 6. Troubleshooting (SSR tasks) This section will help you troubleshoot issues faced while configuring EMS and IBM Elastic Storage System 3200. Note: Ensure that you have gone through the “Prerequisites for installation completion by SSR” on page 62 topic and also followed the steps given in the “Completing the ESS 3200 installation at the customer site (SSR task)”...
  • Page 110 Work with the customer to change the root password again after the installation is complete. • Run essutils manually by typing essutils and press the Enter key. 92  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 111: Ibm Elastic Storage System 3200 Issues

    Note: Change A with B for right canister. If you cannot log in to the BMC with either password, you must contact IBM service. IBM service might recommend to replace a canister or to run a procedure to reset the password.

  • Page 112: Ssr Port Login Issues

    This task requires knowledge of the Linux command line and the vi editor. When you see an interface name, the interface name is mgmt for IBM Elastic Storage System 3200. 1. Log in the IBM Elastic Storage System 3200 or the EMS as a root user.
  • Page 113 4. Edit the ifcfg- file. Press 'i' on the keyboard to enter the insert mode. You can see Insert at bottom of the screen. 5. Modify the IP address and netmask. a. Based on the Installation worksheet information, ensure that you add or modify the IPADDR and NETMASK lines with the correct information.

  • Page 114: Rebooting Bmc

    Enter key. Wait for few minutes and confirm the BMC is healthy again by running: 9. Check FSP interface (EMS or 3200) (HMC1 | BMC) You must get a normal output. 96  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 115: Fixing Console Redirection

    Fixing console redirection When you boot up each 3200 canister for code 20 checkout, you might see the BMC login prompt vs the O/S login prompt. If this is the case you must fix redirection. 1. Login to the BMC by using the following credentials: •...

  • Page 116: Issues In The Console Redirection Enablement Using Ipmitool

    /sbin/ifconfig enP1p8s0f0 Issues in the console redirection enablement using ipmitool IBM SSRs must enable console redirection when logged into the BMC over serial. If a canister is powered on and the password is not working, the ipmitool password could be incorrect.

  • Page 117: Issuing Essutils Commands Manually

    Issuing essutils commands manually IBM SSRs use the essutils commands manually, if they have some issues with the GUI. Complete the following steps to use the essutils commands: ESS 3200 1. Check the storage quickly. /opt/ibm/ess/tools/bin/essstoragequickcheck -N localhost 2. Find missing disks.
  • Page 118 '172.16.0.0/24' ipv4.method manual ipv4.gateway 172.16.0.1 connection.autoconnect yes ; nmcli c reload ; nmcli c up 7. Check the campus interface. /sbin/ifconfig campus 8. Check serviceable events. /usr/sbin/opal-elog-parse -s 9. Gather serviceable event details. 100  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 119: Appendix A. Planning Worksheets (Customer Task)

    Appendix A. Planning worksheets (customer task) Customers are responsible for completing the system planning worksheets. The customer then provides the worksheets to the IBM SSR when they install and configure the system. Installation worksheet This worksheet is intended for customers as part of the TDA process to outline the items that are required to be implemented by the SSR during setup.
  • Page 120 SSR: General guidance • For each IBM Elastic Storage System 3200 canister in the flow (see Chapter 4, “Installing,” on page 45), connect your laptop point-to-point to the SSR port as shown in the SSR access port. You need to use this port to access each canister, perform hardware checks, and set the management IP address.
  • Page 121 Note: The worksheet has default IP addresses filled out in case they were not provided by the customer during the TDA process. Before setting any default IP addresses have the customer confirm that they are not currently in use by the associated networks. Customers must fill in the following values so that the SSR is able perform the required networking tasks.
  • Page 122 This is required only on ESS 3200 and ESS 3500. The default is 101. ESS 3200/3500 VLAN Tag VLAN TAG ESS 5000 Building-Block 1 (lowest position in frame) 104  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 123 IP address Netmask IO node 1 management Interface 192.168.45.21 255.255.255.0 (bottom node in building-block) IO node 1 FSP (HMC1 port) 10.0.0.101 255.255.255.0 interface (bottom node in building-block) IO node 2 management Interface 192.168.45.22 255.255.255.0 (top node in building-block) IO node 2 FSP (HMC1 port) 10.0.0.102 255.255.255.0 interface (top node in building-...
  • Page 124 The Scale license field is only need for ESS 6130 or later. Scale license DAE or DME Note: Write down any vital information that should be shared with the customer / LBS that you encountered during execution of Install Complete. 106  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 125: Appendix B. Switch Vlan Configuration Instructions

    Appendix B. Switch VLAN configuration instructions This topic describes the instructions that are needed to configure an IBM Cumulus switch VLAN. The IBM Cumulus switch would be preconfigured from manufacturing with proper VLAN that includes the following: • Service/FSP/BMC VLAN –...
  • Page 126 Figure 89. RJ45 to serial cable and USB to serial cable • Connect the serial end to the serial to USB cable. • Connect the USB cable to the PC. 108  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 127 Figure 90. USB cable Configure the host PC Configure the host PC as follows: 1. Ensure that the driver for USB to serial cable is connected on the PC. 2. Open the device manager to verify that the COM port is used by the USB to serial cable. 3.
  • Page 128 1. Free up at least two ports (1 IBM Elastic Storage System 3200) on the existing switch. It is better if you can free up a block. Ideally, the current configuration is not scattered where it is easy to convert free ports for IBM Elastic Storage System 3200 usage.
  • Page 129 gateway 192.168.45.1 # EVEN Ports/Lower ports PVID 101 for FSP network auto swp14 iface swp14 bridge-access 101 auto swp16 iface swp16 bridge-access 101 auto swp18 iface swp18 bridge-access 101 auto swp20 iface swp20 bridge-access 101 auto swp22 iface swp22 bridge-access 101 auto swp24 iface swp24 bridge-access 101...
  • Page 130 102 bridge-vids 101 auto swp10 iface swp10 bridge-pvid 102 bridge-vids 101 auto swp11 iface swp11 bridge-pvid 102 bridge-vids 101 auto swp12 iface swp12 bridge-pvid 102 bridge-vids 101 112  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 131 # Bridge setup auto bridge iface bridge bridge-vlan-aware yes bridge-ports glob swp1-48 bridge-pvid 101 bridge-pvid 102 bridge-stp off} Adding additional management switches If the customer is out of ports on a single management switch, then a second one can be used to extend VLANs.
  • Page 132 114  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 133: Appendix C. Dual 24 Port (48 Ports) Mgmt Switch Ess Configuration

    I/O nodes HMC1 ports. • Ports 21 and 22, named as ISL in the orange color, are to be used for ISL between switches only. IBM allows to extend this setup to more than two switches on a line topology, meaning the first and last switch can only have one ISL connection to the following/previous switch.
  • Page 134 Once logged in, apply the switch configuration information to both switches. • Copy and paste the following configuration to the switch and press Enter . • Or, copy the contents of the file in the box (https://ibm.box.com/s/ x3kzg5ykkfsu2t6536um5p0pmbg64xrl) to the switch and press Enter.
  • Page 135 no spanning-tree loopback-detection no shutdown exit interface ethernet 1/13-20 switchport mode access switchport allowed vlan add 101 untagged switchport native vlan 101 switchport allowed vlan remove 1 no shutdown exit interface ethernet 1/21-22 switchport mode hybrid switchport native vlan 100 switchport allowed vlan add 100 untagged switchport allowed vlan add 101 tagged spanning-tree spanning-disabled...
  • Page 136 118  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 137: Appendix D. Cables/Adapters For Ssr Activities

    Note: The figure shows mini USB (ESS 3200), micro USB is need for ESS 3500. Figure 94. Serial (USB mini-B) to USB cable Note: The cable shown in the figure works for ESS 3200/ESS 3500 enclosures. © Copyright IBM Corp. 2021, 2022...
  • Page 138 Figure 95. POWER9 EMS (serial to USB) Figure 96. DB9 to USB cable for power nodes Figure 97. USB extension cable 120  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 139 Figure 98. Adapter to access server via Ethernet Figure 99. Adapter slot to access server via Ethernet Appendix D. Cables/Adapters for SSR activities  121...
  • Page 140 122  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 141: Accessibility Features For The System

    IBM Documentation, and its related publications, are accessibility-enabled. Keyboard navigation This product uses standard Microsoft Windows navigation keys. IBM and accessibility See the IBM Human Ability and Accessibility Center (www.ibm.com/able) for more information about the commitment that IBM has to accessibility. © Copyright IBM Corp. 2021, 2022...
  • Page 142 124  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 143: Glossary

    • See refers you from a non-preferred term to the preferred term or from an abbreviation to the spelled- out form. • See also refers you to a related or contrasting term. For other terms and definitions, see the IBM Terminology website (opens in new window): http://www.ibm.com/software/globalization/terminology building block A pair of servers with shared disk enclosures attached.
  • Page 144 A data structure containing key information about a file system. This information includes the disks assigned to the file system (stripe group), the current state of the file system, and pointers to key files such as quota files and log files. 126  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 145 Hardware Management Console (HMC) Standard interface for configuring and operating partitioned (LPAR) and SMP systems. See Hardware Management Console (HMC). IBM Security Key Lifecycle Manager (ISKLM) For GPFS encryption, the ISKLM is used as an RKM server to store MEKs. independent fileset A fileset that has its own inode space.
  • Page 146 From a system management perspective, it is the central coordinator of the cluster. It also serves as a client node in an ESS building block. master encryption key (MEK) A key that is used to encrypt other keys. See also encryption key. 128  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 147 maximum transmission unit (MTU) The largest packet or frame, specified in octets (eight-bit bytes), that can be sent in a packet- or frame-based network, such as the Internet. The TCP uses the MTU to determine the maximum size of each packet in any transmission. See master encryption key (MEK).
  • Page 148 A network protocol that ensures a loop-free topology for any bridged Ethernet local-area network. The basic function of STP is to prevent bridge loops and the broadcast radiation that results from them. 130  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 149 See secure shell (SSH). See Spanning Tree Protocol (STP). symmetric multiprocessing (SMP) A computer architecture that provides fast performance by making multiple processors available to complete individual processes simultaneously. See Transmission Control Protocol (TCP). Transmission Control Protocol (TCP) A core protocol of the Internet Protocol Suite that provides reliable, ordered, and error-checked delivery of a stream of octets between applications running on hosts communicating over an IP network.
  • Page 150 132  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...

  • Page 151: Index

    31 homologation statement 3 power 31 space 32 resources on web xv IBM Elastic Storage System 3200 acoustic output of drives 38 additional space requirements 35 air temperature without redundant AC power 31 safety notices dimensions and weight 32...
  • Page 152 IBM Elastic Storage System 3200 35 specifications shock 38 vibration 38 SSR access port 59 SSR install 64 SSR port 94 static-sensitive devices 12 submitting xvii trademarks 2 vibration specifications 38 documentation xv resources xv 134  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 153 31 homologation statement 3 power 31 space 32 resources on web xv IBM Elastic Storage System 3200 acoustic output of drives 38 additional space requirements 35 air temperature without redundant AC power 31 safety notices dimensions and weight 32...
  • Page 154 IBM Elastic Storage System 3200 35 specifications shock 38 vibration 38 SSR access port 59 SSR install 64 SSR port 94 static-sensitive devices 12 submitting xvii trademarks 2 vibration specifications 38 documentation xv resources xv 136  IBM Elastic Storage System 3200: Hardware Planning and Installation Guide...
  • Page 156 IBM® Product Number: 5765-DME   5765-DAE SC31-5733-00...

Table of Contents